2024-03-04T12:00:31.078 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T12:00:31.079 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --name vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052/orig.config.yaml 2024-03-04T12:00:31.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T12:00:31.205 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052 branch: wip-vshankar-testing-20240304.042522 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: vshankar@redhat.com first_in_suite: false job_id: '7581052' kernel: client: sha1: distro flavor: default kdb: true sha1: 786939134d7eee4dc7a805245d3f595929af37d7 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240304.042522 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240304.042522 sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3119 sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 sleep_before_teardown: 0 subset: 270/1024 suite: fs suite_branch: wip-vshankar-testing-20240304.042522 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTlTfxDezVwol9bjQQ6jmIeWwFud4Cj37zPliw8SGx9wXnbM3RYPLfGCGV/QOx+pl8qruWfOeQ9sH6ewhO87eI= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEFlv6aDwsRl2DMRULy5B1viRpRIhZ65tMkePrwpwYOcIEqI8YEkqXKAIOj5nPDtOtjM9BAh/96ImtKUlDJShg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_08:26:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T12:00:31.205 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa; will attempt to use it 2024-03-04T12:00:31.206 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks 2024-03-04T12:00:31.206 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T12:00:31.207 INFO:teuthology.task.internal:Checking packages... 2024-03-04T12:00:31.231 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '717ce3cce8d0166e4e577faa8004e6f9cb4128c0' 2024-03-04T12:00:31.231 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T12:00:31.231 INFO:teuthology.packaging:ref: None 2024-03-04T12:00:31.231 INFO:teuthology.packaging:tag: None 2024-03-04T12:00:31.231 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240304.042522 2024-03-04T12:00:31.231 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T12:00:31.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240304.042522 2024-03-04T12:00:31.370 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1845.g717ce3cc 2024-03-04T12:00:31.372 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T12:00:31.378 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T12:00:31.378 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T12:00:31.383 INFO:teuthology.task.internal:Saving configuration 2024-03-04T12:00:31.399 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T12:00:31.404 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T12:00:31.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 11:55:26.342742', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTlTfxDezVwol9bjQQ6jmIeWwFud4Cj37zPliw8SGx9wXnbM3RYPLfGCGV/QOx+pl8qruWfOeQ9sH6ewhO87eI='} 2024-03-04T12:00:31.453 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 11:55:26.341088', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEFlv6aDwsRl2DMRULy5B1viRpRIhZ65tMkePrwpwYOcIEqI8YEkqXKAIOj5nPDtOtjM9BAh/96ImtKUlDJShg='} 2024-03-04T12:00:31.453 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T12:00:31.459 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-04T12:00:31.459 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-04T12:00:31.459 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T12:00:31.532 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c05f6d160>, signals=[15]) 2024-03-04T12:00:31.533 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T12:00:31.539 INFO:teuthology.task.internal:Opening connections... 2024-03-04T12:00:31.540 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:00:31.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:00:31.633 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:00:31.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:00:31.719 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T12:00:31.726 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-03-04T12:00:31.742 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-03-04T12:00:31.743 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-03-04T12:00:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T12:00:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T12:00:31.800 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-03-04T12:00:31.822 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-03-04T12:00:31.838 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-03-04T12:00:31.839 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-03-04T12:00:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-03-04T12:00:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-03-04T12:00:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-03-04T12:00:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T12:00:31.896 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T12:00:31.896 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-03-04T12:00:31.916 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T12:00:31.926 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T12:00:31.932 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T12:00:31.932 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-03-04T12:00:31.935 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-03-04T12:00:31.950 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T12:00:31.957 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T12:00:31.957 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-03-04T12:00:31.991 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-03-04T12:00:32.093 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T12:00:32.147 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T12:00:32.161 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'} 2024-03-04T12:00:32.161 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-04T12:00:32.161 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-04T12:00:32.162 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T12:00:32.162 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T12:00:32.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:00:32.178 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-04T12:00:32.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:00:32.179 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-03-04T12:00:32.235 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-361.el9.x86_64 2024-03-04T12:00:32.236 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-361.el9.x86_64 2024-03-04T12:00:32.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-03-04T12:00:32.237 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-361.el9.x86_64 2024-03-04T12:00:32.237 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.14.0-361.el9.x86_64 2024-03-04T12:00:32.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-03-04T12:00:36.711 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 2.4 MB/s | 6.7 MB 00:02 2024-03-04T12:00:38.472 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 1.5 MB/s | 6.7 MB 00:04 2024-03-04T12:00:39.494 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 1.2 MB/s | 2.0 MB 00:01 2024-03-04T12:00:40.747 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 1.7 MB/s | 2.0 MB 00:01 2024-03-04T12:00:42.705 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-03-04T12:00:43.824 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-03-04T12:00:45.696 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 19 MB 00:01 2024-03-04T12:00:47.006 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-03-04T12:00:52.945 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-03-04T12:00:54.244 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-03-04T12:00:54.481 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-04T12:00:56.259 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2024-03-04T12:01:00.818 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 20 MB 00:04 2024-03-04T12:01:05.626 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-04T12:01:07.858 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-04T12:01:07.858 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-04T12:01:07.969 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-04T12:01:07.979 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:01:07.980 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-03-04T12:01:07.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:01:07.981 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 100 M 2024-03-04T12:01:07.981 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 123 M 2024-03-04T12:01:07.981 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-04T12:01:10.144 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 2.8 MB/s | 6.0 MB 00:02 2024-03-04T12:01:14.219 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 6.3 MB/s | 39 MB 00:06 2024-03-04T12:01:14.844 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 7.1 MB/s | 33 MB 00:04 2024-03-04T12:01:21.425 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 782 kB/s | 20 MB 00:26 2024-03-04T12:01:23.594 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-core-5.14.0-427.el9.x86_64.rpm 1.3 MB/s | 21 MB 00:15 2024-03-04T12:01:23.594 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-04T12:01:23.594 INFO:teuthology.orchestra.run.smithi148.stdout:Total 6.4 MB/s | 100 MB 00:15 2024-03-04T12:01:24.486 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-04T12:01:24.876 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-04T12:01:24.876 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-04T12:01:26.223 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-04T12:01:26.224 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-04T12:01:26.297 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-04T12:01:28.562 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-04T12:01:28.562 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-04T12:01:28.659 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-03-04T12:01:28.663 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout:Install 4 Packages 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:01:28.664 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 100 M 2024-03-04T12:01:28.665 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 123 M 2024-03-04T12:01:28.665 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-03-04T12:01:29.140 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-04T12:01:29.773 INFO:teuthology.orchestra.run.smithi191.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 6.4 MB/s | 6.0 MB 00:00 2024-03-04T12:01:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-04T12:01:29.944 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:01:30.824 INFO:teuthology.orchestra.run.smithi191.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 10 MB/s | 21 MB 00:01 2024-03-04T12:01:31.765 INFO:teuthology.orchestra.run.smithi191.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 13 MB/s | 39 MB 00:02 2024-03-04T12:01:32.140 INFO:teuthology.orchestra.run.smithi191.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 14 MB/s | 33 MB 00:02 2024-03-04T12:01:32.141 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-03-04T12:01:32.141 INFO:teuthology.orchestra.run.smithi191.stdout:Total 29 MB/s | 100 MB 00:03 2024-03-04T12:01:32.721 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:01:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:01:33.024 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-03-04T12:01:33.399 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-03-04T12:01:33.399 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-03-04T12:01:34.748 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-03-04T12:01:34.748 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-03-04T12:01:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:01:37.688 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-03-04T12:01:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:01:38.354 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-04T12:01:38.492 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:01:41.285 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:01:41.462 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:01:42.507 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:01:46.343 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:01:46.523 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:01:51.280 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:11.762 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:11.762 INFO:teuthology.orchestra.run.smithi148.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T12:02:11.762 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:02:11.846 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:12.307 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:12.308 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-04T12:02:12.308 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:02:12.308 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:02:12.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:12.735 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:02:12.735 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:02:12.736 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-04T12:02:13.267 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T12:02:13.413 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:13.414 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T12:02:13.414 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-04T12:02:13.414 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T12:02:13.414 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T12:02:13.414 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-04T12:02:13.414 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T12:02:13.414 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T12:02:13.535 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:13.535 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T12:02:13.535 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 68.el9 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-03-04T12:02:13.579 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8264254 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:02:13.580 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T12:02:13.581 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T12:02:13.581 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T12:02:13.581 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-03-04T12:02:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:02:13.581 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T12:02:13.581 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T12:02:13.582 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-427.el9.x86_64 2024-03-04T12:02:13.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T12:02:14.137 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-03-04T12:02:15.570 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T12:02:15.604 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-03-04T12:02:15.606 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-03-04T12:02:15.622 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.n57eNikXZB 2024-03-04T12:02:15.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.n57eNikXZB 2024-03-04T12:02:15.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.n57eNikXZB 2024-03-04T12:02:15.828 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.n57eNikXZB is 6KB 2024-03-04T12:02:15.876 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.n57eNikXZB 2024-03-04T12:02:15.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T12:02:15.958 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T12:02:15.958 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T12:02:15.958 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T12:02:15.958 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-04T12:02:15.959 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-04T12:02:16.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-03-04T12:02:20.590 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:20.590 INFO:teuthology.orchestra.run.smithi191.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T12:02:20.590 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:02:20.675 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:21.104 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:21.104 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-04T12:02:21.104 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T12:02:21.104 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T12:02:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:02:21.532 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-03-04T12:02:22.087 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T12:02:22.225 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:22.226 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T12:02:22.226 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-04T12:02:22.226 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T12:02:22.226 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T12:02:22.226 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-04T12:02:22.226 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T12:02:22.226 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T12:02:22.347 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T12:02:22.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T12:02:22.348 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2024-03-04T12:02:22.381 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2024-03-04T12:02:22.381 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2024-03-04T12:02:22.381 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.06 2024-03-04T12:02:22.381 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 68.el9 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Group : Unspecified 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 8264254 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : builder@centos.org 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:02:22.382 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T12:02:22.383 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T12:02:22.383 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T12:02:22.383 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2024-03-04T12:02:22.383 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:02:22.383 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T12:02:22.384 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T12:02:22.384 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 5.14.0-427.el9.x86_64 2024-03-04T12:02:22.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T12:02:23.010 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-03-04T12:02:24.382 INFO:teuthology.orchestra.run.smithi191.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T12:02:24.411 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-03-04T12:02:24.413 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-03-04T12:02:24.428 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.8tPax4Egxb 2024-03-04T12:02:24.429 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8tPax4Egxb 2024-03-04T12:02:24.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.8tPax4Egxb 2024-03-04T12:02:29.229 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.8tPax4Egxb is 6KB 2024-03-04T12:02:29.277 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.8tPax4Egxb 2024-03-04T12:02:29.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T12:02:29.361 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T12:02:29.361 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T12:02:29.361 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T12:02:29.361 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-04T12:02:29.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-04T12:02:29.520 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-03-04T12:02:46.128 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-04T12:02:46.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:02:46.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:02:59.536 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-04T12:02:59.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T12:02:59.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:03:04.636 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-04T12:03:13.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:03:13.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:03:16.700 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-04T12:03:28.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:03:28.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:03:59.555 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T12:04:08.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T12:04:08.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:04:09.070 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-03-04T12:04:09.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T12:04:09.511 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-04T12:04:09.512 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-03-04T12:04:09.526 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-427.el9.x86_64 2024-03-04T12:04:09.526 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-04T12:04:09.526 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T12:04:09.526 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T12:04:10.527 DEBUG:teuthology.parallel:result is None 2024-03-04T12:04:28.715 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T12:04:43.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:04:43.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:04:44.115 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-04T12:04:44.519 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:04:44.519 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-04T12:04:44.519 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-04T12:04:44.535 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-427.el9.x86_64 2024-03-04T12:04:44.535 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-04T12:04:44.535 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T12:04:44.535 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T12:04:45.536 DEBUG:teuthology.parallel:result is None 2024-03-04T12:04:45.537 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T12:04:45.545 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T12:04:45.545 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T12:04:45.547 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T12:04:45.568 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T12:04:45.607 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T12:04:45.614 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T12:04:45.614 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T12:04:45.616 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T12:04:45.654 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T12:04:45.661 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T12:04:45.661 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T12:04:45.690 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T12:04:45.798 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T12:04:45.835 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T12:04:45.858 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T12:04:45.888 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T12:04:45.890 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T12:04:45.897 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T12:04:45.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T12:04:45.900 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T12:04:45.931 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T12:04:45.941 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T12:04:45.942 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T12:04:45.972 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T12:04:45.989 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T12:04:46.069 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T12:04:46.122 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T12:04:46.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T12:04:46.249 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:04:46.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T12:04:46.316 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T12:04:46.361 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T12:04:46.423 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T12:04:46.467 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T12:04:46.511 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:04:46.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T12:04:46.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-03-04T12:04:46.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-03-04T12:04:46.631 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T12:04:46.680 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T12:04:46.918 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T12:04:46.929 INFO:teuthology.task.internal:Starting timer... 2024-03-04T12:04:46.930 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T12:04:46.991 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T12:04:47.001 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-04T12:04:47.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-03-04T12:04:47.072 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-03-04T12:04:47.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-03-04T12:04:47.142 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-03-04T12:04:47.145 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T12:04:47.145 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-03-04T12:04:47.171 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-03-04T12:04:47.171 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-03-04T12:04:47.193 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-03-04T12:04:47.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-03-04T12:04:47.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T12:04:47.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:04:47.240 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T12:04:47.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:04:47.268 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T12:04:47.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-03-04T12:04:47.315 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-03-04T12:04:47.344 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T12:04:47.353 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T12:04:47.370 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T12:04:47.371 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T12:09:00.216 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-03-04T12:09:00.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:09:00.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:09:00.309 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-04T12:09:00.390 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-04T12:09:00.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T12:09:00.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T12:09:00.479 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-03-04T12:09:00.554 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T12:09:00.555 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T12:09:00.565 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T12:09:00.565 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T12:09:00.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T12:09:00.568 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T12:09:00.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T12:09:00.595 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T12:09:00.611 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T12:09:00.627 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T12:09:00.642 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-03-04T12:09:00.644 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T12:09:00.655 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-03-04T12:09:00.672 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T12:09:00.673 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-03-04T12:09:00.687 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-03-04T12:09:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T12:09:00.703 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T12:09:00.720 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T12:09:00.750 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-03-04T12:09:00.753 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T12:09:00.753 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-03-04T12:09:00.754 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.754 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.754 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.754 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.769 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.773 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T12:09:00.774 INFO:teuthology.run_tasks:Running task install... 2024-03-04T12:09:00.782 DEBUG:teuthology.task.install:project ceph 2024-03-04T12:09:00.782 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}} 2024-03-04T12:09:00.782 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'} 2024-03-04T12:09:00.783 INFO:teuthology.task.install:Using flavor: default 2024-03-04T12:09:00.789 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T12:09:00.789 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-04T12:09:00.789 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'tag': None, 'wait_for_package': False} 2024-03-04T12:09:00.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T12:09:00.792 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'tag': None, 'wait_for_package': False} 2024-03-04T12:09:00.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T12:09:00.939 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/ 2024-03-04T12:09:00.939 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1845.g717ce3cc 2024-03-04T12:09:00.943 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/ 2024-03-04T12:09:00.943 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1845.g717ce3cc 2024-03-04T12:09:01.048 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T12:09:01.048 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:09:01.048 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T12:09:01.066 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240304.042522/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T12:09:01.066 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:09:01.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T12:09:01.076 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T12:09:01.076 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T12:09:01.093 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T12:09:01.093 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/717ce3cce8d0166e4e577faa8004e6f9cb4128c0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T12:09:01.143 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T12:09:01.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T12:09:01.223 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T12:09:01.240 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T12:09:01.290 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-03-04T12:09:01.291 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-03-04T12:09:01.307 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-03-04T12:09:01.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-03-04T12:09:01.542 INFO:teuthology.orchestra.run.smithi191.stdout:74 files removed 2024-03-04T12:09:01.552 INFO:teuthology.orchestra.run.smithi148.stdout:74 files removed 2024-03-04T12:09:01.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T12:09:01.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T12:09:02.366 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 171 kB/s | 82 kB 00:00 2024-03-04T12:09:02.380 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 169 kB/s | 82 kB 00:00 2024-03-04T12:09:02.780 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-04T12:09:02.808 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-04T12:09:03.231 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-04T12:09:03.250 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-04T12:09:03.792 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-04T12:09:03.870 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-04T12:09:05.217 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-04T12:09:05.348 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-03-04T12:09:07.881 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-03-04T12:09:08.260 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-03-04T12:09:11.752 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-03-04T12:09:12.393 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:03 2024-03-04T12:09:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.1 MB 00:02 2024-03-04T12:09:18.434 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.1 MB 00:02 2024-03-04T12:09:20.197 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-03-04T12:09:20.664 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 15 kB 00:01 2024-03-04T12:09:22.092 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-03-04T12:09:24.025 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 20 MB 00:03 2024-03-04T12:09:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-04T12:09:28.866 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-03-04T12:09:29.684 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T12:09:29.685 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T12:09:29.697 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T12:09:29.698 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T12:09:29.699 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T12:09:29.700 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T12:09:29.701 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T12:09:29.704 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T12:09:29.704 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T12:09:29.788 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 6.6 k 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 5.3 M 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 888 k 2024-03-04T12:09:29.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 145 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 1.5 M 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 138 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 1.8 M 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 7.4 M 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 49 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 9.8 M 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 47 M 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 264 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 749 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 220 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 27 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 18 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T12:09:29.809 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 31 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 745 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 127 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 162 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 320 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 298 k 2024-03-04T12:09:29.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 99 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 85 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.1 M 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 170 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T12:09:29.811 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.4 M 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.1 M 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T12:09:29.812 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 20 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 31 k 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 2.1 M 2024-03-04T12:09:29.813 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 248 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 4.7 M 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 17 M 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 16 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 25 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T12:09:29.814 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 163 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 502 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 5.1 M 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T12:09:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T12:09:29.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 45 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 131 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T12:09:29.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T12:09:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T12:09:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T12:09:29.820 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T12:09:29.821 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout:Install 150 Packages 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 26 Packages 2024-03-04T12:09:29.822 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:09:29.823 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 223 M 2024-03-04T12:09:29.823 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-04T12:09:30.591 INFO:teuthology.orchestra.run.smithi148.stdout:(1/176): ceph-19.0.0-1845.g717ce3cc.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-04T12:09:30.849 INFO:teuthology.orchestra.run.smithi148.stdout:(2/176): ceph-fuse-19.0.0-1845.g717ce3cc.el9.x8 3.4 MB/s | 888 kB 00:00 2024-03-04T12:09:30.918 INFO:teuthology.orchestra.run.smithi148.stdout:(3/176): ceph-immutable-object-cache-19.0.0-184 2.1 MB/s | 145 kB 00:00 2024-03-04T12:09:31.219 INFO:teuthology.orchestra.run.smithi148.stdout:(4/176): ceph-mds-19.0.0-1845.g717ce3cc.el9.x86 7.1 MB/s | 2.1 MB 00:00 2024-03-04T12:09:31.394 INFO:teuthology.orchestra.run.smithi148.stdout:(5/176): ceph-mgr-19.0.0-1845.g717ce3cc.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-04T12:09:31.421 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T12:09:31.422 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T12:09:31.436 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T12:09:31.437 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T12:09:31.438 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T12:09:31.440 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T12:09:31.442 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T12:09:31.444 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T12:09:31.446 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T12:09:31.528 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 6.6 k 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 5.3 M 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 888 k 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 145 k 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 1.5 M 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 138 k 2024-03-04T12:09:31.548 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 1.8 M 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 7.4 M 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 49 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 9.8 M 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 47 M 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 264 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 749 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 220 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 27 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 18 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 31 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 745 k 2024-03-04T12:09:31.549 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 127 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 162 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 320 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 298 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 99 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 85 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.1 M 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 170 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T12:09:31.550 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T12:09:31.551 INFO:teuthology.orchestra.run.smithi191.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.4 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 3.1 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T12:09:31.552 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 20 M 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 31 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 2.1 M 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 248 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 4.7 M 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 17 M 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1845.g717ce3cc.el9 ceph-noarch 16 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 25 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T12:09:31.553 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 163 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 502 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 5.1 M 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T12:09:31.554 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T12:09:31.555 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 45 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.0.0-1845.g717ce3cc.el9 ceph 131 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T12:09:31.556 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T12:09:31.557 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T12:09:31.558 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T12:09:31.559 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-03-04T12:09:31.560 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T12:09:31.563 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:09:31.563 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-03-04T12:09:31.563 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T12:09:31.563 INFO:teuthology.orchestra.run.smithi191.stdout:Install 150 Packages 2024-03-04T12:09:31.563 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 26 Packages 2024-03-04T12:09:31.564 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:09:31.564 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 223 M 2024-03-04T12:09:31.564 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-03-04T12:09:31.980 INFO:teuthology.orchestra.run.smithi148.stdout:(6/176): ceph-mon-19.0.0-1845.g717ce3cc.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-03-04T12:09:32.211 INFO:teuthology.orchestra.run.smithi148.stdout:(7/176): ceph-base-19.0.0-1845.g717ce3cc.el9.x8 3.0 MB/s | 5.3 MB 00:01 2024-03-04T12:09:32.310 INFO:teuthology.orchestra.run.smithi191.stdout:(1/176): ceph-19.0.0-1845.g717ce3cc.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-04T12:09:32.411 INFO:teuthology.orchestra.run.smithi148.stdout:(8/176): ceph-common-19.0.0-1845.g717ce3cc.el9. 10 MB/s | 20 MB 00:01 2024-03-04T12:09:32.478 INFO:teuthology.orchestra.run.smithi148.stdout:(9/176): ceph-selinux-19.0.0-1845.g717ce3cc.el9 379 kB/s | 25 kB 00:00 2024-03-04T12:09:33.010 INFO:teuthology.orchestra.run.smithi191.stdout:(2/176): ceph-fuse-19.0.0-1845.g717ce3cc.el9.x8 1.2 MB/s | 888 kB 00:00 2024-03-04T12:09:33.160 INFO:teuthology.orchestra.run.smithi191.stdout:(3/176): ceph-immutable-object-cache-19.0.0-184 969 kB/s | 145 kB 00:00 2024-03-04T12:09:33.971 INFO:teuthology.orchestra.run.smithi148.stdout:(10/176): ceph-osd-19.0.0-1845.g717ce3cc.el9.x8 8.4 MB/s | 17 MB 00:01 2024-03-04T12:09:34.037 INFO:teuthology.orchestra.run.smithi148.stdout:(11/176): cephfs-mirror-19.0.0-1845.g717ce3cc.e 3.3 MB/s | 220 kB 00:00 2024-03-04T12:09:34.145 INFO:teuthology.orchestra.run.smithi148.stdout:(12/176): cephfs-shell-19.0.0-1845.g717ce3cc.el 250 kB/s | 27 kB 00:00 2024-03-04T12:09:34.221 INFO:teuthology.orchestra.run.smithi148.stdout:(13/176): libcephfs-devel-19.0.0-1845.g717ce3cc 415 kB/s | 31 kB 00:00 2024-03-04T12:09:34.361 INFO:teuthology.orchestra.run.smithi191.stdout:(4/176): ceph-mds-19.0.0-1845.g717ce3cc.el9.x86 1.8 MB/s | 2.1 MB 00:01 2024-03-04T12:09:34.371 INFO:teuthology.orchestra.run.smithi148.stdout:(14/176): libcephfs2-19.0.0-1845.g717ce3cc.el9. 4.9 MB/s | 745 kB 00:00 2024-03-04T12:09:34.446 INFO:teuthology.orchestra.run.smithi148.stdout:(15/176): libcephsqlite-19.0.0-1845.g717ce3cc.e 2.1 MB/s | 163 kB 00:00 2024-03-04T12:09:34.530 INFO:teuthology.orchestra.run.smithi148.stdout:(16/176): librados-devel-19.0.0-1845.g717ce3cc. 1.5 MB/s | 127 kB 00:00 2024-03-04T12:09:34.640 INFO:teuthology.orchestra.run.smithi148.stdout:(17/176): libradosstriper1-19.0.0-1845.g717ce3c 4.5 MB/s | 502 kB 00:00 2024-03-04T12:09:35.222 INFO:teuthology.orchestra.run.smithi148.stdout:(18/176): ceph-radosgw-19.0.0-1845.g717ce3cc.el 3.2 MB/s | 9.8 MB 00:03 2024-03-04T12:09:35.227 INFO:teuthology.orchestra.run.smithi191.stdout:(5/176): ceph-mgr-19.0.0-1845.g717ce3cc.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-03-04T12:09:35.306 INFO:teuthology.orchestra.run.smithi148.stdout:(19/176): librgw2-19.0.0-1845.g717ce3cc.el9.x86 7.7 MB/s | 5.1 MB 00:00 2024-03-04T12:09:35.330 INFO:teuthology.orchestra.run.smithi148.stdout:(20/176): python3-ceph-argparse-19.0.0-1845.g71 416 kB/s | 45 kB 00:00 2024-03-04T12:09:35.381 INFO:teuthology.orchestra.run.smithi148.stdout:(21/176): python3-ceph-common-19.0.0-1845.g717c 1.7 MB/s | 131 kB 00:00 2024-03-04T12:09:35.448 INFO:teuthology.orchestra.run.smithi148.stdout:(22/176): python3-cephfs-19.0.0-1845.g717ce3cc. 1.4 MB/s | 162 kB 00:00 2024-03-04T12:09:35.473 INFO:teuthology.orchestra.run.smithi148.stdout:(23/176): python3-rados-19.0.0-1845.g717ce3cc.e 3.4 MB/s | 320 kB 00:00 2024-03-04T12:09:35.599 INFO:teuthology.orchestra.run.smithi148.stdout:(24/176): python3-rgw-19.0.0-1845.g717ce3cc.el9 793 kB/s | 99 kB 00:00 2024-03-04T12:09:36.007 INFO:teuthology.orchestra.run.smithi148.stdout:(25/176): ceph-test-19.0.0-1845.g717ce3cc.el9.x 13 MB/s | 47 MB 00:03 2024-03-04T12:09:36.032 INFO:teuthology.orchestra.run.smithi148.stdout:(26/176): python3-rbd-19.0.0-1845.g717ce3cc.el9 510 kB/s | 298 kB 00:00 2024-03-04T12:09:36.058 INFO:teuthology.orchestra.run.smithi148.stdout:(27/176): rbd-fuse-19.0.0-1845.g717ce3cc.el9.x8 186 kB/s | 85 kB 00:00 2024-03-04T12:09:36.133 INFO:teuthology.orchestra.run.smithi148.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1845.g 419 kB/s | 31 kB 00:00 2024-03-04T12:09:36.192 INFO:teuthology.orchestra.run.smithi148.stdout:(29/176): rbd-nbd-19.0.0-1845.g717ce3cc.el9.x86 1.0 MB/s | 170 kB 00:00 2024-03-04T12:09:36.217 INFO:teuthology.orchestra.run.smithi148.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1845.g717ce3c 1.6 MB/s | 138 kB 00:00 2024-03-04T12:09:36.259 INFO:teuthology.orchestra.run.smithi148.stdout:(31/176): rbd-mirror-19.0.0-1845.g717ce3cc.el9. 12 MB/s | 3.1 MB 00:00 2024-03-04T12:09:36.334 INFO:teuthology.orchestra.run.smithi148.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1845.g71 3.2 MB/s | 248 kB 00:00 2024-03-04T12:09:36.401 INFO:teuthology.orchestra.run.smithi148.stdout:(33/176): ceph-mgr-rook-19.0.0-1845.g717ce3cc.e 741 kB/s | 49 kB 00:00 2024-03-04T12:09:36.419 INFO:teuthology.orchestra.run.smithi191.stdout:(6/176): ceph-mon-19.0.0-1845.g717ce3cc.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-03-04T12:09:36.468 INFO:teuthology.orchestra.run.smithi148.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1845.g7 246 kB/s | 16 kB 00:00 2024-03-04T12:09:36.552 INFO:teuthology.orchestra.run.smithi148.stdout:(35/176): ceph-volume-19.0.0-1845.g717ce3cc.el9 3.1 MB/s | 264 kB 00:00 2024-03-04T12:09:36.652 INFO:teuthology.orchestra.run.smithi148.stdout:(36/176): cephadm-19.0.0-1845.g717ce3cc.el9.noa 7.3 MB/s | 749 kB 00:00 2024-03-04T12:09:36.702 INFO:teuthology.orchestra.run.smithi148.stdout:(37/176): ceph-mgr-dashboard-19.0.0-1845.g717ce 3.5 MB/s | 1.8 MB 00:00 2024-03-04T12:09:36.728 INFO:teuthology.orchestra.run.smithi148.stdout:(38/176): cephfs-top-19.0.0-1845.g717ce3cc.el9. 242 kB/s | 18 kB 00:00 2024-03-04T12:09:36.819 INFO:teuthology.orchestra.run.smithi148.stdout:(39/176): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-03-04T12:09:37.045 INFO:teuthology.orchestra.run.smithi148.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 106 kB 00:00 2024-03-04T12:09:37.195 INFO:teuthology.orchestra.run.smithi148.stdout:(41/176): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 920 kB 00:00 2024-03-04T12:09:37.248 INFO:teuthology.orchestra.run.smithi148.stdout:(42/176): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2024-03-04T12:09:37.253 INFO:teuthology.orchestra.run.smithi191.stdout:(7/176): ceph-base-19.0.0-1845.g717ce3cc.el9.x8 1.0 MB/s | 5.3 MB 00:05 2024-03-04T12:09:37.270 INFO:teuthology.orchestra.run.smithi148.stdout:(43/176): elfutils-libelf-devel-0.190-2.el9.x86 91 kB/s | 20 kB 00:00 2024-03-04T12:09:37.329 INFO:teuthology.orchestra.run.smithi148.stdout:(44/176): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 307 kB 00:00 2024-03-04T12:09:37.354 INFO:teuthology.orchestra.run.smithi148.stdout:(45/176): flexiblas-3.0.4-8.el9.x86_64.rpm 380 kB/s | 32 kB 00:00 2024-03-04T12:09:37.646 INFO:teuthology.orchestra.run.smithi148.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 54 kB/s | 17 kB 00:00 2024-03-04T12:09:37.671 INFO:teuthology.orchestra.run.smithi148.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 32 kB/s | 10 kB 00:00 2024-03-04T12:09:37.904 INFO:teuthology.orchestra.run.smithi191.stdout:(8/176): ceph-common-19.0.0-1845.g717ce3cc.el9. 3.4 MB/s | 20 MB 00:05 2024-03-04T12:09:37.954 INFO:teuthology.orchestra.run.smithi191.stdout:(9/176): ceph-selinux-19.0.0-1845.g717ce3cc.el9 505 kB/s | 25 kB 00:00 2024-03-04T12:09:38.055 INFO:teuthology.orchestra.run.smithi148.stdout:(48/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-03-04T12:09:38.080 INFO:teuthology.orchestra.run.smithi148.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 108 kB/s | 44 kB 00:00 2024-03-04T12:09:38.097 INFO:teuthology.orchestra.run.smithi148.stdout:(50/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 374 kB/s | 159 kB 00:00 2024-03-04T12:09:38.273 INFO:teuthology.orchestra.run.smithi148.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 661 kB 00:00 2024-03-04T12:09:38.364 INFO:teuthology.orchestra.run.smithi148.stdout:(52/176): libtool-2.4.6-45.el9.x86_64.rpm 2.0 MB/s | 578 kB 00:00 2024-03-04T12:09:38.390 INFO:teuthology.orchestra.run.smithi148.stdout:(53/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 83 kB/s | 24 kB 00:00 2024-03-04T12:09:38.415 INFO:teuthology.orchestra.run.smithi148.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 328 kB/s | 46 kB 00:00 2024-03-04T12:09:38.473 INFO:teuthology.orchestra.run.smithi148.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2024-03-04T12:09:38.499 INFO:teuthology.orchestra.run.smithi148.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 411 kB/s | 34 kB 00:00 2024-03-04T12:09:38.524 INFO:teuthology.orchestra.run.smithi148.stdout:(57/176): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-04T12:09:38.988 INFO:teuthology.orchestra.run.smithi191.stdout:(10/176): ceph-osd-19.0.0-1845.g717ce3cc.el9.x8 6.5 MB/s | 17 MB 00:02 2024-03-04T12:09:39.054 INFO:teuthology.orchestra.run.smithi191.stdout:(11/176): cephfs-mirror-19.0.0-1845.g717ce3cc.e 3.2 MB/s | 220 kB 00:00 2024-03-04T12:09:39.097 INFO:teuthology.orchestra.run.smithi191.stdout:(12/176): cephfs-shell-19.0.0-1845.g717ce3cc.el 646 kB/s | 27 kB 00:00 2024-03-04T12:09:39.130 INFO:teuthology.orchestra.run.smithi191.stdout:(13/176): libcephfs-devel-19.0.0-1845.g717ce3cc 937 kB/s | 31 kB 00:00 2024-03-04T12:09:39.239 INFO:teuthology.orchestra.run.smithi191.stdout:(14/176): libcephfs2-19.0.0-1845.g717ce3cc.el9. 6.7 MB/s | 745 kB 00:00 2024-03-04T12:09:39.289 INFO:teuthology.orchestra.run.smithi191.stdout:(15/176): libcephsqlite-19.0.0-1845.g717ce3cc.e 3.2 MB/s | 163 kB 00:00 2024-03-04T12:09:39.332 INFO:teuthology.orchestra.run.smithi191.stdout:(16/176): librados-devel-19.0.0-1845.g717ce3cc. 3.0 MB/s | 127 kB 00:00 2024-03-04T12:09:39.424 INFO:teuthology.orchestra.run.smithi191.stdout:(17/176): libradosstriper1-19.0.0-1845.g717ce3c 5.3 MB/s | 502 kB 00:00 2024-03-04T12:09:39.541 INFO:teuthology.orchestra.run.smithi148.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 1.5 MB/s | 1.5 MB 00:01 2024-03-04T12:09:40.033 INFO:teuthology.orchestra.run.smithi148.stdout:(59/176): openblas-openmp-0.3.21-2.el9.x86_64.r 3.0 MB/s | 4.7 MB 00:01 2024-03-04T12:09:40.057 INFO:teuthology.orchestra.run.smithi191.stdout:(18/176): librgw2-19.0.0-1845.g717ce3cc.el9.x86 8.1 MB/s | 5.1 MB 00:00 2024-03-04T12:09:40.083 INFO:teuthology.orchestra.run.smithi148.stdout:(60/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.9 MB/s | 3.0 MB 00:01 2024-03-04T12:09:40.108 INFO:teuthology.orchestra.run.smithi191.stdout:(19/176): python3-ceph-argparse-19.0.0-1845.g71 908 kB/s | 45 kB 00:00 2024-03-04T12:09:40.116 INFO:teuthology.orchestra.run.smithi148.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 1.4 MB/s | 811 kB 00:00 2024-03-04T12:09:40.141 INFO:teuthology.orchestra.run.smithi148.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-04T12:09:40.158 INFO:teuthology.orchestra.run.smithi191.stdout:(20/176): python3-ceph-common-19.0.0-1845.g717c 2.6 MB/s | 131 kB 00:00 2024-03-04T12:09:40.209 INFO:teuthology.orchestra.run.smithi191.stdout:(21/176): python3-cephfs-19.0.0-1845.g717ce3cc. 3.2 MB/s | 162 kB 00:00 2024-03-04T12:09:40.276 INFO:teuthology.orchestra.run.smithi191.stdout:(22/176): python3-rados-19.0.0-1845.g717ce3cc.e 4.7 MB/s | 320 kB 00:00 2024-03-04T12:09:40.343 INFO:teuthology.orchestra.run.smithi191.stdout:(23/176): python3-rbd-19.0.0-1845.g717ce3cc.el9 4.4 MB/s | 298 kB 00:00 2024-03-04T12:09:40.350 INFO:teuthology.orchestra.run.smithi148.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-03-04T12:09:40.393 INFO:teuthology.orchestra.run.smithi191.stdout:(24/176): python3-rgw-19.0.0-1845.g717ce3cc.el9 1.9 MB/s | 99 kB 00:00 2024-03-04T12:09:40.444 INFO:teuthology.orchestra.run.smithi191.stdout:(25/176): rbd-fuse-19.0.0-1845.g717ce3cc.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-04T12:09:40.467 INFO:teuthology.orchestra.run.smithi148.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 239 kB/s | 78 kB 00:00 2024-03-04T12:09:40.794 INFO:teuthology.orchestra.run.smithi191.stdout:(26/176): rbd-mirror-19.0.0-1845.g717ce3cc.el9. 8.8 MB/s | 3.1 MB 00:00 2024-03-04T12:09:40.809 INFO:teuthology.orchestra.run.smithi148.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 665 kB/s | 227 kB 00:00 2024-03-04T12:09:40.844 INFO:teuthology.orchestra.run.smithi191.stdout:(27/176): rbd-nbd-19.0.0-1845.g717ce3cc.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-04T12:09:40.886 INFO:teuthology.orchestra.run.smithi191.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1845.g 756 kB/s | 31 kB 00:00 2024-03-04T12:09:40.937 INFO:teuthology.orchestra.run.smithi191.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1845.g717ce3c 2.7 MB/s | 138 kB 00:00 2024-03-04T12:09:41.154 INFO:teuthology.orchestra.run.smithi191.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1845.g717ce 8.2 MB/s | 1.8 MB 00:00 2024-03-04T12:09:41.168 INFO:teuthology.orchestra.run.smithi148.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 68 kB/s | 24 kB 00:00 2024-03-04T12:09:41.326 INFO:teuthology.orchestra.run.smithi148.stdout:(67/176): postgresql-server-13.11-1.el9.x86_64. 4.7 MB/s | 5.7 MB 00:01 2024-03-04T12:09:41.468 INFO:teuthology.orchestra.run.smithi148.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 60 kB/s | 18 kB 00:00 2024-03-04T12:09:41.543 INFO:teuthology.orchestra.run.smithi148.stdout:(69/176): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 5.8 MB 00:01 2024-03-04T12:09:41.569 INFO:teuthology.orchestra.run.smithi148.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 598 kB/s | 144 kB 00:00 2024-03-04T12:09:41.585 INFO:teuthology.orchestra.run.smithi148.stdout:(71/176): python3-markupsafe-1.1.1-12.el9.x86_6 270 kB/s | 32 kB 00:00 2024-03-04T12:09:41.871 INFO:teuthology.orchestra.run.smithi191.stdout:(31/176): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-03-04T12:09:41.877 INFO:teuthology.orchestra.run.smithi148.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 747 kB/s | 217 kB 00:00 2024-03-04T12:09:41.902 INFO:teuthology.orchestra.run.smithi148.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 551 kB/s | 174 kB 00:00 2024-03-04T12:09:41.930 INFO:teuthology.orchestra.run.smithi191.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1845.g71 4.1 MB/s | 248 kB 00:00 2024-03-04T12:09:42.014 INFO:teuthology.orchestra.run.smithi191.stdout:(33/176): ceph-mgr-rook-19.0.0-1845.g717ce3cc.e 591 kB/s | 49 kB 00:00 2024-03-04T12:09:42.056 INFO:teuthology.orchestra.run.smithi191.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1845.g7 396 kB/s | 16 kB 00:00 2024-03-04T12:09:42.111 INFO:teuthology.orchestra.run.smithi148.stdout:(74/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.4 MB/s | 4.8 MB 00:00 2024-03-04T12:09:42.114 INFO:teuthology.orchestra.run.smithi191.stdout:(35/176): ceph-volume-19.0.0-1845.g717ce3cc.el9 4.4 MB/s | 264 kB 00:00 2024-03-04T12:09:42.128 INFO:teuthology.orchestra.run.smithi148.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 275 kB/s | 69 kB 00:00 2024-03-04T12:09:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 165 kB/s | 41 kB 00:00 2024-03-04T12:09:42.187 INFO:teuthology.orchestra.run.smithi148.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-04T12:09:42.223 INFO:teuthology.orchestra.run.smithi191.stdout:(36/176): cephadm-19.0.0-1845.g717ce3cc.el9.noa 6.8 MB/s | 749 kB 00:00 2024-03-04T12:09:42.271 INFO:teuthology.orchestra.run.smithi148.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 132 kB 00:00 2024-03-04T12:09:42.273 INFO:teuthology.orchestra.run.smithi191.stdout:(37/176): cephfs-top-19.0.0-1845.g717ce3cc.el9. 364 kB/s | 18 kB 00:00 2024-03-04T12:09:42.296 INFO:teuthology.orchestra.run.smithi148.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 211 kB 00:00 2024-03-04T12:09:42.321 INFO:teuthology.orchestra.run.smithi148.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 353 kB/s | 47 kB 00:00 2024-03-04T12:09:42.346 INFO:teuthology.orchestra.run.smithi148.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 573 kB/s | 43 kB 00:00 2024-03-04T12:09:42.388 INFO:teuthology.orchestra.run.smithi148.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 581 kB/s | 39 kB 00:00 2024-03-04T12:09:42.655 INFO:teuthology.orchestra.run.smithi148.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 41 kB 00:00 2024-03-04T12:09:42.680 INFO:teuthology.orchestra.run.smithi148.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 740 kB/s | 215 kB 00:00 2024-03-04T12:09:42.939 INFO:teuthology.orchestra.run.smithi148.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 119 kB/s | 34 kB 00:00 2024-03-04T12:09:42.964 INFO:teuthology.orchestra.run.smithi148.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 153 kB/s | 43 kB 00:00 2024-03-04T12:09:42.974 INFO:teuthology.orchestra.run.smithi191.stdout:(38/176): ceph-test-19.0.0-1845.g717ce3cc.el9.x 9.4 MB/s | 47 MB 00:05 2024-03-04T12:09:43.082 INFO:teuthology.orchestra.run.smithi191.stdout:(39/176): automake-1.16.2-8.el9.noarch.rpm 819 kB/s | 662 kB 00:00 2024-03-04T12:09:43.224 INFO:teuthology.orchestra.run.smithi191.stdout:(40/176): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 920 kB 00:00 2024-03-04T12:09:43.250 INFO:teuthology.orchestra.run.smithi191.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 635 kB/s | 106 kB 00:00 2024-03-04T12:09:43.256 INFO:teuthology.orchestra.run.smithi148.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 194 kB/s | 56 kB 00:00 2024-03-04T12:09:43.281 INFO:teuthology.orchestra.run.smithi148.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 877 kB/s | 300 kB 00:00 2024-03-04T12:09:43.292 INFO:teuthology.orchestra.run.smithi191.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 490 kB/s | 20 kB 00:00 2024-03-04T12:09:43.317 INFO:teuthology.orchestra.run.smithi191.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-04T12:09:43.343 INFO:teuthology.orchestra.run.smithi191.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 633 kB/s | 32 kB 00:00 2024-03-04T12:09:43.490 INFO:teuthology.orchestra.run.smithi148.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 374 kB/s | 87 kB 00:00 2024-03-04T12:09:43.501 INFO:teuthology.orchestra.run.smithi191.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-04T12:09:43.527 INFO:teuthology.orchestra.run.smithi191.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 93 kB/s | 17 kB 00:00 2024-03-04T12:09:43.552 INFO:teuthology.orchestra.run.smithi191.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 199 kB/s | 10 kB 00:00 2024-03-04T12:09:43.582 INFO:teuthology.orchestra.run.smithi148.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 209 kB/s | 63 kB 00:00 2024-03-04T12:09:43.586 INFO:teuthology.orchestra.run.smithi191.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-03-04T12:09:43.611 INFO:teuthology.orchestra.run.smithi191.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 750 kB/s | 44 kB 00:00 2024-03-04T12:09:43.662 INFO:teuthology.orchestra.run.smithi191.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-04T12:09:43.695 INFO:teuthology.orchestra.run.smithi191.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2024-03-04T12:09:43.721 INFO:teuthology.orchestra.run.smithi191.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 422 kB/s | 24 kB 00:00 2024-03-04T12:09:43.746 INFO:teuthology.orchestra.run.smithi191.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 918 kB/s | 46 kB 00:00 2024-03-04T12:09:43.772 INFO:teuthology.orchestra.run.smithi191.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-04T12:09:43.797 INFO:teuthology.orchestra.run.smithi191.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-04T12:09:43.822 INFO:teuthology.orchestra.run.smithi191.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 683 kB/s | 34 kB 00:00 2024-03-04T12:09:43.865 INFO:teuthology.orchestra.run.smithi148.stdout:(91/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-03-04T12:09:43.890 INFO:teuthology.orchestra.run.smithi148.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 110 kB/s | 44 kB 00:00 2024-03-04T12:09:43.916 INFO:teuthology.orchestra.run.smithi148.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 236 kB/s | 79 kB 00:00 2024-03-04T12:09:43.941 INFO:teuthology.orchestra.run.smithi148.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-03-04T12:09:43.999 INFO:teuthology.orchestra.run.smithi148.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 194 kB 00:00 2024-03-04T12:09:44.023 INFO:teuthology.orchestra.run.smithi191.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-03-04T12:09:44.041 INFO:teuthology.orchestra.run.smithi148.stdout:(96/176): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 802 kB 00:00 2024-03-04T12:09:44.067 INFO:teuthology.orchestra.run.smithi148.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2024-03-04T12:09:44.073 INFO:teuthology.orchestra.run.smithi191.stdout:(58/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-03-04T12:09:44.092 INFO:teuthology.orchestra.run.smithi148.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-03-04T12:09:44.107 INFO:teuthology.orchestra.run.smithi191.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 19 MB/s | 1.5 MB 00:00 2024-03-04T12:09:44.149 INFO:teuthology.orchestra.run.smithi191.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-04T12:09:44.174 INFO:teuthology.orchestra.run.smithi191.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-03-04T12:09:44.176 INFO:teuthology.orchestra.run.smithi148.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.1 MB 00:00 2024-03-04T12:09:44.193 INFO:teuthology.orchestra.run.smithi148.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-04T12:09:44.218 INFO:teuthology.orchestra.run.smithi148.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 999 kB/s | 124 kB 00:00 2024-03-04T12:09:44.260 INFO:teuthology.orchestra.run.smithi148.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.3 MB/s | 113 kB 00:00 2024-03-04T12:09:44.302 INFO:teuthology.orchestra.run.smithi148.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 2.2 MB/s | 187 kB 00:00 2024-03-04T12:09:44.369 INFO:teuthology.orchestra.run.smithi148.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-04T12:09:44.392 INFO:teuthology.orchestra.run.smithi191.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-03-04T12:09:44.425 INFO:teuthology.orchestra.run.smithi191.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-03-04T12:09:44.451 INFO:teuthology.orchestra.run.smithi191.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-04T12:09:44.560 INFO:teuthology.orchestra.run.smithi148.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-03-04T12:09:44.601 INFO:teuthology.orchestra.run.smithi191.stdout:(65/176): ceph-radosgw-19.0.0-1845.g717ce3cc.el 1.3 MB/s | 9.8 MB 00:07 2024-03-04T12:09:44.693 INFO:teuthology.orchestra.run.smithi191.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-04T12:09:44.719 INFO:teuthology.orchestra.run.smithi191.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 848 kB/s | 227 kB 00:00 2024-03-04T12:09:44.736 INFO:teuthology.orchestra.run.smithi148.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-04T12:09:44.744 INFO:teuthology.orchestra.run.smithi191.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 174 kB/s | 24 kB 00:00 2024-03-04T12:09:44.761 INFO:teuthology.orchestra.run.smithi148.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-03-04T12:09:44.769 INFO:teuthology.orchestra.run.smithi191.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 241 kB/s | 18 kB 00:00 2024-03-04T12:09:44.803 INFO:teuthology.orchestra.run.smithi191.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 540 kB/s | 32 kB 00:00 2024-03-04T12:09:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 293 kB/s | 19 kB 00:00 2024-03-04T12:09:44.828 INFO:teuthology.orchestra.run.smithi191.stdout:(71/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-04T12:09:44.995 INFO:teuthology.orchestra.run.smithi191.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-04T12:09:45.020 INFO:teuthology.orchestra.run.smithi191.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-04T12:09:45.046 INFO:teuthology.orchestra.run.smithi191.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 808 kB/s | 174 kB 00:00 2024-03-04T12:09:45.071 INFO:teuthology.orchestra.run.smithi191.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 912 kB/s | 69 kB 00:00 2024-03-04T12:09:45.097 INFO:teuthology.orchestra.run.smithi191.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 817 kB/s | 41 kB 00:00 2024-03-04T12:09:45.122 INFO:teuthology.orchestra.run.smithi191.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-04T12:09:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-04T12:09:45.198 INFO:teuthology.orchestra.run.smithi191.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 630 kB/s | 47 kB 00:00 2024-03-04T12:09:45.265 INFO:teuthology.orchestra.run.smithi191.stdout:(80/176): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 211 kB 00:00 2024-03-04T12:09:45.290 INFO:teuthology.orchestra.run.smithi191.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 300 kB/s | 43 kB 00:00 2024-03-04T12:09:45.520 INFO:teuthology.orchestra.run.smithi148.stdout:(109/176): python3-devel-3.9.17-2.el9.x86_64.rp 188 kB/s | 245 kB 00:01 2024-03-04T12:09:45.824 INFO:teuthology.orchestra.run.smithi191.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 39 kB 00:00 2024-03-04T12:09:45.849 INFO:teuthology.orchestra.run.smithi191.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 73 kB/s | 41 kB 00:00 2024-03-04T12:09:45.962 INFO:teuthology.orchestra.run.smithi148.stdout:(110/176): libarrow-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-03-04T12:09:45.978 INFO:teuthology.orchestra.run.smithi148.stdout:(111/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 22 kB/s | 26 kB 00:01 2024-03-04T12:09:45.995 INFO:teuthology.orchestra.run.smithi148.stdout:(112/176): liboath-2.6.7-2.el9.x86_64.rpm 102 kB/s | 49 kB 00:00 2024-03-04T12:09:46.008 INFO:teuthology.orchestra.run.smithi191.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-03-04T12:09:46.021 INFO:teuthology.orchestra.run.smithi148.stdout:(113/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-04T12:09:46.033 INFO:teuthology.orchestra.run.smithi191.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 1.0 MB/s | 215 kB 00:00 2024-03-04T12:09:46.058 INFO:teuthology.orchestra.run.smithi191.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 162 kB/s | 34 kB 00:00 2024-03-04T12:09:46.071 INFO:teuthology.orchestra.run.smithi148.stdout:(114/176): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-04T12:09:46.084 INFO:teuthology.orchestra.run.smithi191.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 580 kB/s | 43 kB 00:00 2024-03-04T12:09:46.109 INFO:teuthology.orchestra.run.smithi191.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-04T12:09:46.143 INFO:teuthology.orchestra.run.smithi191.stdout:(89/176): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-04T12:09:46.168 INFO:teuthology.orchestra.run.smithi191.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-03-04T12:09:46.193 INFO:teuthology.orchestra.run.smithi191.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 747 kB/s | 63 kB 00:00 2024-03-04T12:09:46.219 INFO:teuthology.orchestra.run.smithi191.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 583 kB/s | 44 kB 00:00 2024-03-04T12:09:46.236 INFO:teuthology.orchestra.run.smithi191.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-04T12:09:46.270 INFO:teuthology.orchestra.run.smithi191.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-03-04T12:09:46.320 INFO:teuthology.orchestra.run.smithi191.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 802 kB 00:00 2024-03-04T12:09:46.370 INFO:teuthology.orchestra.run.smithi191.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 194 kB 00:00 2024-03-04T12:09:46.395 INFO:teuthology.orchestra.run.smithi191.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2024-03-04T12:09:46.421 INFO:teuthology.orchestra.run.smithi191.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2024-03-04T12:09:46.454 INFO:teuthology.orchestra.run.smithi148.stdout:(115/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 838 kB 00:00 2024-03-04T12:09:46.455 INFO:teuthology.orchestra.run.smithi191.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-03-04T12:09:46.505 INFO:teuthology.orchestra.run.smithi191.stdout:(100/176): python3-cryptography-36.0.1-4.el9.x8 10 MB/s | 1.1 MB 00:00 2024-03-04T12:09:46.522 INFO:teuthology.orchestra.run.smithi191.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.2 MB/s | 124 kB 00:00 2024-03-04T12:09:46.548 INFO:teuthology.orchestra.run.smithi191.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.2 MB/s | 113 kB 00:00 2024-03-04T12:09:46.573 INFO:teuthology.orchestra.run.smithi191.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 2.7 MB/s | 187 kB 00:00 2024-03-04T12:09:46.646 INFO:teuthology.orchestra.run.smithi148.stdout:(116/176): python3-asyncssh-2.13.2-4.el9.noarch 876 kB/s | 548 kB 00:00 2024-03-04T12:09:46.672 INFO:teuthology.orchestra.run.smithi148.stdout:(117/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 72 kB/s | 43 kB 00:00 2024-03-04T12:09:46.697 INFO:teuthology.orchestra.run.smithi148.stdout:(118/176): python3-cachetools-4.2.4-1.el9.noarc 133 kB/s | 32 kB 00:00 2024-03-04T12:09:46.722 INFO:teuthology.orchestra.run.smithi148.stdout:(119/176): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-03-04T12:09:46.723 INFO:teuthology.orchestra.run.smithi191.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-03-04T12:09:46.822 INFO:teuthology.orchestra.run.smithi148.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-04T12:09:46.881 INFO:teuthology.orchestra.run.smithi148.stdout:(121/176): python3-cheroot-8.6.0-4.el9.noarch.r 936 kB/s | 172 kB 00:00 2024-03-04T12:09:47.007 INFO:teuthology.orchestra.run.smithi191.stdout:(105/176): lua-devel-5.4.4-4.el9.x86_64.rpm 49 kB/s | 22 kB 00:00 2024-03-04T12:09:47.064 INFO:teuthology.orchestra.run.smithi148.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 886 kB/s | 303 kB 00:00 2024-03-04T12:09:47.090 INFO:teuthology.orchestra.run.smithi148.stdout:(123/176): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-04T12:09:47.132 INFO:teuthology.orchestra.run.smithi148.stdout:(124/176): python3-google-auth-2.28.0-1.el9.noa 809 kB/s | 203 kB 00:00 2024-03-04T12:09:47.399 INFO:teuthology.orchestra.run.smithi191.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 811 kB/s | 548 kB 00:00 2024-03-04T12:09:47.491 INFO:teuthology.orchestra.run.smithi191.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-03-04T12:09:47.525 INFO:teuthology.orchestra.run.smithi191.stdout:(108/176): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-03-04T12:09:47.549 INFO:teuthology.orchestra.run.smithi148.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-03-04T12:09:47.590 INFO:teuthology.orchestra.run.smithi148.stdout:(126/176): python3-grpcio-tools-1.46.7-10.el9.x 289 kB/s | 144 kB 00:00 2024-03-04T12:09:47.615 INFO:teuthology.orchestra.run.smithi148.stdout:(127/176): python3-jaraco-8.2.1-3.el9.noarch.rp 22 kB/s | 11 kB 00:00 2024-03-04T12:09:47.625 INFO:teuthology.orchestra.run.smithi191.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-03-04T12:09:47.641 INFO:teuthology.orchestra.run.smithi148.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-03-04T12:09:47.666 INFO:teuthology.orchestra.run.smithi148.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-04T12:09:47.691 INFO:teuthology.orchestra.run.smithi148.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-04T12:09:47.717 INFO:teuthology.orchestra.run.smithi148.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-04T12:09:47.734 INFO:teuthology.orchestra.run.smithi191.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-03-04T12:09:47.843 INFO:teuthology.orchestra.run.smithi191.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-03-04T12:09:47.909 INFO:teuthology.orchestra.run.smithi148.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-03-04T12:09:47.927 INFO:teuthology.orchestra.run.smithi191.stdout:(112/176): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-04T12:09:47.943 INFO:teuthology.orchestra.run.smithi148.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-03-04T12:09:47.960 INFO:teuthology.orchestra.run.smithi191.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-04T12:09:47.967 INFO:teuthology.orchestra.run.smithi148.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 315 kB/s | 79 kB 00:00 2024-03-04T12:09:47.993 INFO:teuthology.orchestra.run.smithi148.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-04T12:09:48.036 INFO:teuthology.orchestra.run.smithi191.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-03-04T12:09:48.060 INFO:teuthology.orchestra.run.smithi148.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-04T12:09:48.085 INFO:teuthology.orchestra.run.smithi148.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-03-04T12:09:48.110 INFO:teuthology.orchestra.run.smithi148.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 771 kB/s | 90 kB 00:00 2024-03-04T12:09:48.135 INFO:teuthology.orchestra.run.smithi148.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-04T12:09:48.136 INFO:teuthology.orchestra.run.smithi191.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-03-04T12:09:48.161 INFO:teuthology.orchestra.run.smithi148.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-03-04T12:09:48.161 INFO:teuthology.orchestra.run.smithi191.stdout:(116/176): python3-asyncssh-2.13.2-4.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-03-04T12:09:48.186 INFO:teuthology.orchestra.run.smithi148.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-04T12:09:48.211 INFO:teuthology.orchestra.run.smithi148.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-04T12:09:48.228 INFO:teuthology.orchestra.run.smithi191.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-03-04T12:09:48.236 INFO:teuthology.orchestra.run.smithi148.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-04T12:09:48.254 INFO:teuthology.orchestra.run.smithi191.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-04T12:09:48.261 INFO:teuthology.orchestra.run.smithi148.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-03-04T12:09:48.303 INFO:teuthology.orchestra.run.smithi148.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-04T12:09:48.329 INFO:teuthology.orchestra.run.smithi148.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 979 kB/s | 90 kB 00:00 2024-03-04T12:09:48.329 INFO:teuthology.orchestra.run.smithi191.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-04T12:09:48.372 INFO:teuthology.orchestra.run.smithi191.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-04T12:09:48.421 INFO:teuthology.orchestra.run.smithi148.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-03-04T12:09:48.430 INFO:teuthology.orchestra.run.smithi191.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-03-04T12:09:48.446 INFO:teuthology.orchestra.run.smithi148.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-03-04T12:09:48.464 INFO:teuthology.orchestra.run.smithi191.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-03-04T12:09:48.471 INFO:teuthology.orchestra.run.smithi148.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-04T12:09:48.531 INFO:teuthology.orchestra.run.smithi191.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-04T12:09:48.640 INFO:teuthology.orchestra.run.smithi191.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-04T12:09:48.698 INFO:teuthology.orchestra.run.smithi191.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-03-04T12:09:48.732 INFO:teuthology.orchestra.run.smithi191.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-03-04T12:09:48.738 INFO:teuthology.orchestra.run.smithi148.stdout:(150/176): librados2-19.0.0-1845.g717ce3cc.el9. 12 MB/s | 3.4 MB 00:00 2024-03-04T12:09:48.788 INFO:teuthology.orchestra.run.smithi148.stdout:(151/176): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-03-04T12:09:48.791 INFO:teuthology.orchestra.run.smithi191.stdout:(127/176): protobuf-compiler-3.14.0-13.el9.x86_ 389 kB/s | 863 kB 00:02 2024-03-04T12:09:48.813 INFO:teuthology.orchestra.run.smithi148.stdout:(152/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 907 kB/s | 68 kB 00:00 2024-03-04T12:09:48.816 INFO:teuthology.orchestra.run.smithi191.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 150 kB/s | 18 kB 00:00 2024-03-04T12:09:48.842 INFO:teuthology.orchestra.run.smithi191.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-03-04T12:09:48.909 INFO:teuthology.orchestra.run.smithi191.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-04T12:09:48.976 INFO:teuthology.orchestra.run.smithi191.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-03-04T12:09:49.001 INFO:teuthology.orchestra.run.smithi191.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-03-04T12:09:49.030 INFO:teuthology.orchestra.run.smithi148.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 8.3 MB/s | 1.8 MB 00:00 2024-03-04T12:09:49.043 INFO:teuthology.orchestra.run.smithi191.stdout:(133/176): python3-jaraco-functools-3.5.0-2.el9 77 kB/s | 19 kB 00:00 2024-03-04T12:09:49.069 INFO:teuthology.orchestra.run.smithi191.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 851 kB/s | 79 kB 00:00 2024-03-04T12:09:49.072 INFO:teuthology.orchestra.run.smithi148.stdout:(154/176): NetworkManager-1.46.0-1.el9.x86_64.r 8.8 MB/s | 2.3 MB 00:00 2024-03-04T12:09:49.097 INFO:teuthology.orchestra.run.smithi148.stdout:(155/176): NetworkManager-team-1.46.0-1.el9.x86 631 kB/s | 41 kB 00:00 2024-03-04T12:09:49.102 INFO:teuthology.orchestra.run.smithi191.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 572 kB/s | 58 kB 00:00 2024-03-04T12:09:49.131 INFO:teuthology.orchestra.run.smithi148.stdout:(156/176): NetworkManager-tui-1.46.0-1.el9.x86_ 4.1 MB/s | 247 kB 00:00 2024-03-04T12:09:49.156 INFO:teuthology.orchestra.run.smithi148.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-03-04T12:09:49.169 INFO:teuthology.orchestra.run.smithi191.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-03-04T12:09:49.182 INFO:teuthology.orchestra.run.smithi148.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 705 kB/s | 35 kB 00:00 2024-03-04T12:09:49.195 INFO:teuthology.orchestra.run.smithi191.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-03-04T12:09:49.207 INFO:teuthology.orchestra.run.smithi148.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-03-04T12:09:49.232 INFO:teuthology.orchestra.run.smithi148.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-04T12:09:49.262 INFO:teuthology.orchestra.run.smithi191.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-03-04T12:09:49.282 INFO:teuthology.orchestra.run.smithi148.stdout:(161/176): librbd1-19.0.0-1845.g717ce3cc.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-03-04T12:09:49.287 INFO:teuthology.orchestra.run.smithi191.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-04T12:09:49.316 INFO:teuthology.orchestra.run.smithi148.stdout:(162/176): iproute-6.7.0-2.el9.x86_64.rpm 7.2 MB/s | 802 kB 00:00 2024-03-04T12:09:49.341 INFO:teuthology.orchestra.run.smithi148.stdout:(163/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 3.9 MB/s | 433 kB 00:00 2024-03-04T12:09:49.363 INFO:teuthology.orchestra.run.smithi191.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-04T12:09:49.366 INFO:teuthology.orchestra.run.smithi148.stdout:(164/176): libacl-2.3.1-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-03-04T12:09:49.388 INFO:teuthology.orchestra.run.smithi191.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-03-04T12:09:49.392 INFO:teuthology.orchestra.run.smithi148.stdout:(165/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-03-04T12:09:49.417 INFO:teuthology.orchestra.run.smithi148.stdout:(166/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 154 kB 00:00 2024-03-04T12:09:49.430 INFO:teuthology.orchestra.run.smithi191.stdout:(142/176): python3-pecan-1.4.2-3.el9.noarch.rpm 704 kB/s | 272 kB 00:00 2024-03-04T12:09:49.442 INFO:teuthology.orchestra.run.smithi148.stdout:(167/176): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 135 kB 00:00 2024-03-04T12:09:49.456 INFO:teuthology.orchestra.run.smithi191.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 386 kB/s | 36 kB 00:00 2024-03-04T12:09:49.468 INFO:teuthology.orchestra.run.smithi148.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 555 kB/s | 28 kB 00:00 2024-03-04T12:09:49.481 INFO:teuthology.orchestra.run.smithi191.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 550 kB/s | 51 kB 00:00 2024-03-04T12:09:49.493 INFO:teuthology.orchestra.run.smithi148.stdout:(169/176): libsmartcols-2.37.4-18.el9.x86_64.rp 835 kB/s | 63 kB 00:00 2024-03-04T12:09:49.535 INFO:teuthology.orchestra.run.smithi148.stdout:(170/176): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-04T12:09:49.548 INFO:teuthology.orchestra.run.smithi191.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 974 kB/s | 90 kB 00:00 2024-03-04T12:09:49.573 INFO:teuthology.orchestra.run.smithi191.stdout:(146/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-03-04T12:09:49.599 INFO:teuthology.orchestra.run.smithi191.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-04T12:09:49.627 INFO:teuthology.orchestra.run.smithi148.stdout:(171/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-04T12:09:49.641 INFO:teuthology.orchestra.run.smithi191.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-03-04T12:09:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-04T12:09:49.710 INFO:teuthology.orchestra.run.smithi191.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-04T12:09:49.727 INFO:teuthology.orchestra.run.smithi148.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 2.2 MB/s | 438 kB 00:00 2024-03-04T12:09:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-04T12:09:49.752 INFO:teuthology.orchestra.run.smithi148.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 245 kB/s | 30 kB 00:00 2024-03-04T12:09:49.775 INFO:teuthology.orchestra.run.smithi191.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T12:09:49.778 INFO:teuthology.orchestra.run.smithi148.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-04T12:09:49.828 INFO:teuthology.orchestra.run.smithi148.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-03-04T12:09:49.834 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-04T12:09:49.834 INFO:teuthology.orchestra.run.smithi148.stdout:Total 11 MB/s | 223 MB 00:20 2024-03-04T12:09:49.876 INFO:teuthology.orchestra.run.smithi191.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-03-04T12:09:49.883 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-04T12:09:50.035 INFO:teuthology.orchestra.run.smithi191.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-03-04T12:09:50.068 INFO:teuthology.orchestra.run.smithi191.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-04T12:09:50.110 INFO:teuthology.orchestra.run.smithi191.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-04T12:09:50.160 INFO:teuthology.orchestra.run.smithi191.stdout:(156/176): librados2-19.0.0-1845.g717ce3cc.el9. 6.5 MB/s | 3.4 MB 00:00 2024-03-04T12:09:50.186 INFO:teuthology.orchestra.run.smithi191.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 911 kB/s | 69 kB 00:00 2024-03-04T12:09:50.196 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-04T12:09:50.196 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-04T12:09:50.211 INFO:teuthology.orchestra.run.smithi191.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 703 kB/s | 35 kB 00:00 2024-03-04T12:09:50.237 INFO:teuthology.orchestra.run.smithi191.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-03-04T12:09:50.270 INFO:teuthology.orchestra.run.smithi191.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2024-03-04T12:09:50.304 INFO:teuthology.orchestra.run.smithi191.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-03-04T12:09:50.338 INFO:teuthology.orchestra.run.smithi191.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.4 MB/s | 433 kB 00:00 2024-03-04T12:09:50.372 INFO:teuthology.orchestra.run.smithi191.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 329 kB/s | 22 kB 00:00 2024-03-04T12:09:50.397 INFO:teuthology.orchestra.run.smithi191.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 107 kB 00:00 2024-03-04T12:09:50.423 INFO:teuthology.orchestra.run.smithi191.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-03-04T12:09:50.439 INFO:teuthology.orchestra.run.smithi191.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 135 kB 00:00 2024-03-04T12:09:50.465 INFO:teuthology.orchestra.run.smithi191.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-03-04T12:09:50.482 INFO:teuthology.orchestra.run.smithi191.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 658 kB/s | 28 kB 00:00 2024-03-04T12:09:50.524 INFO:teuthology.orchestra.run.smithi191.stdout:(169/176): openssl-3.0.7-27.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-03-04T12:09:50.583 INFO:teuthology.orchestra.run.smithi191.stdout:(170/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 22 MB/s | 2.2 MB 00:00 2024-03-04T12:09:50.625 INFO:teuthology.orchestra.run.smithi191.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 22 MB/s | 2.2 MB 00:00 2024-03-04T12:09:50.659 INFO:teuthology.orchestra.run.smithi191.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-03-04T12:09:50.684 INFO:teuthology.orchestra.run.smithi191.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 519 kB/s | 30 kB 00:00 2024-03-04T12:09:50.718 INFO:teuthology.orchestra.run.smithi191.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-04T12:09:50.752 INFO:teuthology.orchestra.run.smithi191.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-04T12:09:51.560 INFO:teuthology.orchestra.run.smithi191.stdout:(176/176): librbd1-19.0.0-1845.g717ce3cc.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-03-04T12:09:51.564 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-03-04T12:09:51.564 INFO:teuthology.orchestra.run.smithi191.stdout:Total 11 MB/s | 223 MB 00:20 2024-03-04T12:09:51.611 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-03-04T12:09:51.929 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-03-04T12:09:51.930 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-03-04T12:09:52.160 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-04T12:09:52.160 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-04T12:09:53.899 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-03-04T12:09:53.899 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-03-04T12:09:59.316 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-04T12:09:59.432 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T12:09:59.459 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T12:09:59.505 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T12:09:59.533 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T12:09:59.592 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T12:09:59.631 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T12:09:59.685 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T12:09:59.717 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T12:09:59.743 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T12:09:59.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T12:10:00.136 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T12:10:00.203 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:00.332 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:00.362 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:00.451 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:00.481 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 13/202 2024-03-04T12:10:00.636 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T12:10:00.691 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T12:10:00.719 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:00.822 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:00.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 17/202 2024-03-04T12:10:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-03-04T12:10:00.910 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 17/202 2024-03-04T12:10:00.949 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T12:10:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T12:10:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T12:10:00.993 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T12:10:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T12:10:01.081 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T12:10:01.210 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T12:10:01.247 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T12:10:01.277 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T12:10:01.307 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T12:10:01.378 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T12:10:01.424 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T12:10:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T12:10:01.492 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T12:10:01.539 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T12:10:01.564 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T12:10:01.586 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T12:10:01.616 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T12:10:01.646 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T12:10:01.757 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:01.823 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:01.946 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T12:10:01.989 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:02.031 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:02.119 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T12:10:02.148 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:02.228 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:02.248 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T12:10:02.257 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 13/202 2024-03-04T12:10:02.278 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T12:10:02.347 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T12:10:02.401 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T12:10:02.420 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T12:10:02.431 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T12:10:02.466 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T12:10:02.475 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T12:10:02.502 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:02.506 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T12:10:02.570 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T12:10:02.605 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T12:10:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 17/202 2024-03-04T12:10:02.650 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T12:10:02.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T12:10:02.702 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 17/202 2024-03-04T12:10:02.710 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T12:10:02.732 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T12:10:02.755 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T12:10:02.770 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T12:10:02.783 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T12:10:02.797 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9. 39/202 2024-03-04T12:10:02.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el 40/202 2024-03-04T12:10:02.862 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_6 41/202 2024-03-04T12:10:02.931 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T12:10:02.963 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T12:10:02.977 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T12:10:02.995 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T12:10:03.026 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T12:10:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T12:10:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T12:10:03.222 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T12:10:03.278 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T12:10:03.354 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T12:10:03.397 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T12:10:03.501 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:03.556 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:03.771 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T12:10:03.799 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T12:10:03.875 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T12:10:03.878 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T12:10:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T12:10:03.964 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T12:10:03.995 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T12:10:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T12:10:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T12:10:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T12:10:04.062 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T12:10:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_6 52/202 2024-03-04T12:10:04.117 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T12:10:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T12:10:04.148 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T12:10:04.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T12:10:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T12:10:04.235 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T12:10:04.286 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T12:10:04.295 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T12:10:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T12:10:04.365 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T12:10:04.399 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T12:10:04.426 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T12:10:04.495 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T12:10:04.531 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9. 39/202 2024-03-04T12:10:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T12:10:04.574 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el 40/202 2024-03-04T12:10:04.603 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_6 41/202 2024-03-04T12:10:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T12:10:04.708 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T12:10:04.717 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T12:10:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T12:10:04.790 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T12:10:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T12:10:04.931 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T12:10:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T12:10:04.995 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T12:10:05.027 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T12:10:05.081 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T12:10:05.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T12:10:05.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T12:10:05.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T12:10:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T12:10:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T12:10:05.442 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T12:10:05.484 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T12:10:05.509 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T12:10:05.581 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T12:10:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T12:10:05.664 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T12:10:05.678 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T12:10:05.694 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T12:10:05.735 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T12:10:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T12:10:05.744 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T12:10:05.781 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T12:10:05.819 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_6 52/202 2024-03-04T12:10:05.838 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T12:10:05.852 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T12:10:05.878 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T12:10:05.918 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T12:10:05.944 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T12:10:05.947 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T12:10:05.968 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T12:10:06.011 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T12:10:06.047 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T12:10:06.169 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T12:10:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T12:10:06.281 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T12:10:06.326 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T12:10:06.374 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T12:10:06.430 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T12:10:06.450 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T12:10:06.530 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T12:10:06.573 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T12:10:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T12:10:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T12:10:06.715 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T12:10:06.776 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T12:10:06.792 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T12:10:06.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T12:10:06.831 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T12:10:06.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T12:10:06.893 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T12:10:06.904 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T12:10:06.928 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T12:10:06.936 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T12:10:06.959 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T12:10:06.972 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T12:10:06.987 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T12:10:07.019 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T12:10:07.021 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T12:10:07.052 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T12:10:07.055 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T12:10:07.083 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T12:10:07.098 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T12:10:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T12:10:07.140 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T12:10:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T12:10:07.240 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T12:10:07.274 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T12:10:07.274 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T12:10:07.315 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T12:10:07.329 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T12:10:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T12:10:07.359 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T12:10:07.400 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T12:10:07.438 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T12:10:07.494 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T12:10:07.521 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T12:10:07.534 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T12:10:07.572 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T12:10:07.574 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T12:10:07.600 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T12:10:07.603 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T12:10:07.625 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T12:10:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T12:10:07.701 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T12:10:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T12:10:07.797 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T12:10:07.825 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T12:10:07.932 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T12:10:07.974 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T12:10:08.022 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T12:10:08.079 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T12:10:08.263 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T12:10:08.311 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T12:10:08.440 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T12:10:08.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T12:10:08.510 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T12:10:08.529 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T12:10:08.541 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T12:10:08.577 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T12:10:08.601 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T12:10:08.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T12:10:08.632 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T12:10:08.658 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T12:10:08.660 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T12:10:08.695 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T12:10:08.734 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T12:10:08.738 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T12:10:08.766 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T12:10:08.770 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T12:10:08.795 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T12:10:08.872 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T12:10:08.906 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T12:10:08.939 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T12:10:08.960 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T12:10:08.980 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T12:10:08.984 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T12:10:09.015 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T12:10:09.063 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T12:10:09.103 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T12:10:09.186 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T12:10:09.238 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T12:10:09.269 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T12:10:09.316 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T12:10:09.413 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T12:10:09.462 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T12:10:09.700 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T12:10:09.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T12:10:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T12:10:10.243 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T12:10:10.277 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T12:10:10.325 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T12:10:10.412 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T12:10:10.453 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T12:10:10.631 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T12:10:10.659 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T12:10:10.737 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T12:10:11.035 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T12:10:11.104 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T12:10:11.131 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T12:10:11.384 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T12:10:11.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T12:10:11.478 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 125/202 2024-03-04T12:10:11.498 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T12:10:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 125/202 2024-03-04T12:10:11.599 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 126/202 2024-03-04T12:10:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T12:10:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T12:10:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 129/202 2024-03-04T12:10:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 129/202 2024-03-04T12:10:12.216 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 130/202 2024-03-04T12:10:12.715 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T12:10:12.813 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T12:10:12.839 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T12:10:13.154 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T12:10:13.177 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 125/202 2024-03-04T12:10:13.252 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 125/202 2024-03-04T12:10:13.282 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 126/202 2024-03-04T12:10:13.311 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T12:10:13.767 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T12:10:14.014 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 129/202 2024-03-04T12:10:14.286 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 129/202 2024-03-04T12:10:14.316 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 130/202 2024-03-04T12:10:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:15.016 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 132/202 2024-03-04T12:10:15.107 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 132/202 2024-03-04T12:10:15.107 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T12:10:15.108 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T12:10:15.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:15.141 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 133/202 2024-03-04T12:10:16.480 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:16.520 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:17.064 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 131/202 2024-03-04T12:10:17.099 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 132/202 2024-03-04T12:10:17.190 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 132/202 2024-03-04T12:10:17.191 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T12:10:17.191 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T12:10:17.191 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:17.233 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 133/202 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 133/202 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-03-04T12:10:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:26.721 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 134/202 2024-03-04T12:10:26.736 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 134/202 2024-03-04T12:10:26.737 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:26.737 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T12:10:26.737 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:28.199 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 135/202 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 135/202 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T12:10:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 136/202 2024-03-04T12:10:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 136/202 2024-03-04T12:10:28.455 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:28.455 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T12:10:28.455 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T12:10:28.455 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T12:10:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 137/202 2024-03-04T12:10:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 137/202 2024-03-04T12:10:28.923 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:28.923 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T12:10:28.923 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T12:10:28.923 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T12:10:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:29.010 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T12:10:29.419 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 139/202 2024-03-04T12:10:29.450 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 139/202 2024-03-04T12:10:29.482 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.e 140/202 2024-03-04T12:10:29.563 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc. 141/202 2024-03-04T12:10:29.590 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 142/202 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 133/202 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-03-04T12:10:29.615 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:29.781 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 134/202 2024-03-04T12:10:29.797 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 134/202 2024-03-04T12:10:29.797 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:29.797 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T12:10:29.797 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:29.818 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 142/202 2024-03-04T12:10:29.840 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 143/202 2024-03-04T12:10:30.810 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 143/202 2024-03-04T12:10:30.833 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 144/202 2024-03-04T12:10:30.933 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 144/202 2024-03-04T12:10:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el 145/202 2024-03-04T12:10:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 146/202 2024-03-04T12:10:31.149 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 146/202 2024-03-04T12:10:31.150 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:31.150 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T12:10:31.150 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T12:10:31.150 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T12:10:31.150 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:31.194 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 147/202 2024-03-04T12:10:31.205 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 147/202 2024-03-04T12:10:31.260 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 148/202 2024-03-04T12:10:31.285 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 135/202 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 135/202 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T12:10:31.310 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:31.518 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 136/202 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 136/202 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T12:10:31.541 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:31.986 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 137/202 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 137/202 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T12:10:32.009 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:32.096 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T12:10:32.123 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T12:10:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 150/202 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 150/202 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T12:10:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:32.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1845.g717ce 151/202 2024-03-04T12:10:32.244 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1845.g717ce 151/202 2024-03-04T12:10:32.244 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:32.244 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T12:10:32.244 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:32.289 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 152/202 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 152/202 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T12:10:32.311 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:32.555 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 139/202 2024-03-04T12:10:32.586 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 139/202 2024-03-04T12:10:32.617 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.e 140/202 2024-03-04T12:10:32.622 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:32.646 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:32.646 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:32.646 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T12:10:32.646 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T12:10:32.647 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T12:10:32.647 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:32.677 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc. 141/202 2024-03-04T12:10:32.700 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 142/202 2024-03-04T12:10:32.928 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 142/202 2024-03-04T12:10:32.951 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 143/202 2024-03-04T12:10:33.920 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 143/202 2024-03-04T12:10:33.952 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 144/202 2024-03-04T12:10:34.052 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 144/202 2024-03-04T12:10:34.225 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el 145/202 2024-03-04T12:10:34.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 146/202 2024-03-04T12:10:34.276 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 146/202 2024-03-04T12:10:34.277 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:34.277 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T12:10:34.277 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T12:10:34.277 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T12:10:34.277 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:34.321 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 147/202 2024-03-04T12:10:34.332 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 147/202 2024-03-04T12:10:34.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 148/202 2024-03-04T12:10:35.240 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T12:10:35.273 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 150/202 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 150/202 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T12:10:35.296 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:35.340 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1845.g717ce 151/202 2024-03-04T12:10:35.362 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1845.g717ce 151/202 2024-03-04T12:10:35.362 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:35.362 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T12:10:35.362 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:35.408 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 152/202 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 152/202 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T12:10:35.430 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:35.734 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T12:10:35.756 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:37.094 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 154/202 2024-03-04T12:10:37.127 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 155/202 2024-03-04T12:10:37.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 156/202 2024-03-04T12:10:37.184 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T12:10:37.367 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 158/202 2024-03-04T12:10:37.483 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T12:10:37.524 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 160/202 2024-03-04T12:10:37.576 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T12:10:37.624 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T12:10:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T12:10:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T12:10:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T12:10:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_ 166/202 2024-03-04T12:10:37.804 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T12:10:37.868 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T12:10:37.918 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T12:10:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T12:10:38.515 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:38.551 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:38.571 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 172/202 2024-03-04T12:10:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T12:10:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T12:10:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T12:10:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T12:10:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T12:10:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T12:10:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T12:10:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T12:10:39.794 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T12:10:39.813 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T12:10:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T12:10:39.843 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T12:10:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T12:10:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T12:10:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T12:10:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T12:10:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T12:10:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T12:10:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T12:10:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T12:10:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T12:10:40.213 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 154/202 2024-03-04T12:10:40.219 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T12:10:40.244 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T12:10:40.253 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 155/202 2024-03-04T12:10:40.270 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T12:10:40.284 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 156/202 2024-03-04T12:10:40.295 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T12:10:40.302 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:40.302 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:40.311 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T12:10:40.481 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:40.482 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T12:10:40.502 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 158/202 2024-03-04T12:10:40.564 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T12:10:40.596 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T12:10:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T12:10:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T12:10:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T12:10:40.659 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 160/202 2024-03-04T12:10:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T12:10:40.709 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T12:10:40.739 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T12:10:40.768 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T12:10:40.840 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T12:10:40.886 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T12:10:40.930 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T12:10:40.961 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_ 166/202 2024-03-04T12:10:40.990 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T12:10:41.062 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T12:10:41.126 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T12:10:41.155 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T12:10:41.772 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:41.811 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:41.831 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T12:10:42.656 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 172/202 2024-03-04T12:10:42.731 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T12:10:42.772 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T12:10:42.864 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T12:10:42.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T12:10:42.927 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T12:10:42.952 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T12:10:42.978 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T12:10:42.978 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T12:10:43.046 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T12:10:43.074 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T12:10:43.113 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T12:10:43.113 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T12:10:43.198 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T12:10:43.224 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T12:10:43.257 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T12:10:43.290 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T12:10:43.318 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T12:10:43.350 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T12:10:43.385 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T12:10:43.440 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T12:10:43.472 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T12:10:43.497 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T12:10:43.522 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T12:10:43.547 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T12:10:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T12:10:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:43.767 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T12:10:43.767 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T12:10:43.842 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T12:10:43.874 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T12:10:43.900 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T12:10:43.925 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T12:10:43.951 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T12:10:44.016 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 1/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 2/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 3/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 4/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1845.g717ce 5/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 6/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 7/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 8/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 9/202 2024-03-04T12:10:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 10/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 13/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 14/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_ 15/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 17/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_6 18/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 19/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 20/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el 21/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9. 22/202 2024-03-04T12:10:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_6 23/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 24/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 25/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 26/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 27/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 28/202 2024-03-04T12:10:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 29/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc. 30/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 31/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 32/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 33/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el 34/202 2024-03-04T12:10:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 35/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.e 36/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 37/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 38/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 39/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T12:10:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T12:10:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T12:10:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T12:10:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T12:10:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T12:10:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T12:10:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T12:10:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 151/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T12:10:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T12:10:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T12:10:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T12:10:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T12:10:48.542 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T12:10:48.543 INFO:teuthology.orchestra.run.smithi148.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:48.544 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.546 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T12:10:48.547 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.6-16.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.548 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T12:10:48.549 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T12:10:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T12:10:48.551 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T12:10:48.552 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T12:10:48.553 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T12:10:48.554 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T12:10:48.555 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T12:10:48.556 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T12:10:48.556 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T12:10:48.556 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:48.556 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-04T12:10:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T12:10:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 1/202 2024-03-04T12:10:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 2/202 2024-03-04T12:10:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 3/202 2024-03-04T12:10:49.050 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 4/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1845.g717ce 5/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 6/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 7/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 8/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 9/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 10/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 11/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 12/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 13/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 14/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_ 15/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 16/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 17/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_6 18/202 2024-03-04T12:10:49.051 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86 19/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 20/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el 21/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9. 22/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_6 23/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 24/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 25/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 26/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 27/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 28/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 29/202 2024-03-04T12:10:49.052 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc. 30/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noa 31/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.n 32/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1845.g717 33/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el 34/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 35/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.e 36/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 37/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 38/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 39/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T12:10:49.067 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T12:10:49.068 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T12:10:49.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T12:10:49.070 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T12:10:49.071 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T12:10:49.072 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T12:10:49.073 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T12:10:49.074 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 151/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 153/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T12:10:49.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T12:10:49.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T12:10:49.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T12:10:49.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T12:10:49.190 DEBUG:teuthology.parallel:result is None 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:51.951 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.952 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T12:10:51.953 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.0.0-1845.g717ce3cc.el9.noarch 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T12:10:51.955 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-16.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T12:10:51.956 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T12:10:51.957 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T12:10:51.958 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T12:10:51.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T12:10:51.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T12:10:51.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T12:10:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T12:10:51.963 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.0.0-1845.g717ce3cc.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T12:10:51.964 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:51.965 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-03-04T12:10:52.483 DEBUG:teuthology.parallel:result is None 2024-03-04T12:10:52.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T12:10:52.625 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T12:10:52.650 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-1845.g717ce3cc.el9 2024-03-04T12:10:52.651 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1845.g717ce3cc.el9 2024-03-04T12:10:52.651 INFO:teuthology.task.install:The correct ceph version 19.0.0-1845.g717ce3cc is installed. 2024-03-04T12:10:52.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T12:10:52.789 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T12:10:52.811 INFO:teuthology.orchestra.run.smithi191.stdout:19.0.0-1845.g717ce3cc.el9 2024-03-04T12:10:52.811 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1845.g717ce3cc.el9 2024-03-04T12:10:52.812 INFO:teuthology.task.install:The correct ceph version 19.0.0-1845.g717ce3cc is installed. 2024-03-04T12:10:52.813 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T12:10:52.813 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:52.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T12:10:52.848 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:52.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T12:10:52.882 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T12:10:52.882 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:52.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T12:10:52.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T12:10:52.986 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:52.987 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T12:10:53.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T12:10:53.082 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T12:10:53.082 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:53.082 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T12:10:53.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T12:10:53.180 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:53.180 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T12:10:53.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T12:10:53.274 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T12:10:53.274 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:53.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T12:10:53.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T12:10:53.369 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:53.369 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T12:10:53.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T12:10:53.462 INFO:teuthology.run_tasks:Running task ceph... 2024-03-04T12:10:53.543 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-04T12:10:53.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-03-04T12:10:53.545 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2024-03-04T12:10:53.572 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-04T12:10:53.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T12:10:53.613 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T12:10:53.639 INFO:tasks.ceph:Creating extra log directories... 2024-03-04T12:10:53.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T12:10:53.680 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T12:10:53.713 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-04T12:10:53.713 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-04T12:10:53.714 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-04T12:10:53.714 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:53.714 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-04T12:10:53.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-04T12:10:53.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-04T12:10:53.900 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:53.901 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-04T12:10:53.915 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-04T12:10:54.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-04T12:10:54.064 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-04T12:10:54.064 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-04T12:10:54.064 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581052', 'branch': 'wip-vshankar-testing-20240304.042522', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7581052', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240304.042522'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240304.042522', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3119, 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'sleep_before_teardown': 0, 'subset': '270/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240304.042522', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTlTfxDezVwol9bjQQ6jmIeWwFud4Cj37zPliw8SGx9wXnbM3RYPLfGCGV/QOx+pl8qruWfOeQ9sH6ewhO87eI=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEFlv6aDwsRl2DMRULy5B1viRpRIhZ65tMkePrwpwYOcIEqI8YEkqXKAIOj5nPDtOtjM9BAh/96ImtKUlDJShg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-03-04_08:26:39', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-04T12:10:54.065 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T12:10:54.067 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T12:10:54.113 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T12:10:54.121 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T12:10:54.179 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:54.179 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T12:10:54.201 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T12:10:54.201 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-03-04T12:10:54.256 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-04 12:10:41.070301196 +0000 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-04 12:08:37.784629117 +0000 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-04 12:08:37.784629117 +0000 2024-03-04T12:10:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-04 12:08:37.784629117 +0000 2024-03-04T12:10:54.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T12:10:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-04T12:10:54.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-04T12:10:54.323 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000259504 s, 2.0 MB/s 2024-03-04T12:10:54.324 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T12:10:54.380 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-04 12:10:41.070301196 +0000 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-04 12:08:38.136620359 +0000 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-04 12:08:38.136620359 +0000 2024-03-04T12:10:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-04 12:08:38.136620359 +0000 2024-03-04T12:10:54.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T12:10:54.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-04T12:10:54.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-04T12:10:54.500 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.0002155 s, 2.4 MB/s 2024-03-04T12:10:54.501 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T12:10:54.557 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-03-04T12:10:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-04 12:10:41.071301172 +0000 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-04 12:08:38.485611675 +0000 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-04 12:08:38.485611675 +0000 2024-03-04T12:10:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-04 12:08:38.485611675 +0000 2024-03-04T12:10:54.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T12:10:54.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-04T12:10:54.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-04T12:10:54.678 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000227904 s, 2.2 MB/s 2024-03-04T12:10:54.680 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T12:10:54.735 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-04 12:10:41.071301172 +0000 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-04 12:08:38.842602791 +0000 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-04 12:08:38.842602791 +0000 2024-03-04T12:10:54.791 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-04 12:08:38.842602791 +0000 2024-03-04T12:10:54.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T12:10:54.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-04T12:10:54.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-04T12:10:54.856 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000262888 s, 1.9 MB/s 2024-03-04T12:10:54.857 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T12:10:54.912 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:54.912 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:54.913 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T12:10:54.929 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T12:10:54.929 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-03-04T12:10:54.985 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T12:10:54.985 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:54.985 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-04T12:10:54.985 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:54.986 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:54.986 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 12:10:44.346699570 +0000 2024-03-04T12:10:54.986 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 12:08:37.668509125 +0000 2024-03-04T12:10:54.986 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 12:08:37.668509125 +0000 2024-03-04T12:10:54.986 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 12:08:37.668509125 +0000 2024-03-04T12:10:54.986 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T12:10:55.052 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T12:10:55.052 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T12:10:55.052 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000246796 s, 2.1 MB/s 2024-03-04T12:10:55.053 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T12:10:55.109 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-03-04T12:10:55.164 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 12:10:44.347699553 +0000 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 12:08:38.007503354 +0000 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 12:08:38.007503354 +0000 2024-03-04T12:10:55.165 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 12:08:38.007503354 +0000 2024-03-04T12:10:55.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T12:10:55.231 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T12:10:55.231 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T12:10:55.232 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00019301 s, 2.7 MB/s 2024-03-04T12:10:55.233 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T12:10:55.289 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-03-04T12:10:55.345 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T12:10:55.345 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 12:10:44.347699553 +0000 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 12:08:38.353497463 +0000 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 12:08:38.353497463 +0000 2024-03-04T12:10:55.346 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 12:08:38.353497463 +0000 2024-03-04T12:10:55.346 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T12:10:55.411 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T12:10:55.411 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T12:10:55.411 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000200742 s, 2.6 MB/s 2024-03-04T12:10:55.412 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T12:10:55.468 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 12:10:44.347699553 +0000 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 12:08:38.686491794 +0000 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 12:08:38.686491794 +0000 2024-03-04T12:10:55.523 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 12:08:38.686491794 +0000 2024-03-04T12:10:55.524 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T12:10:55.588 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T12:10:55.588 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T12:10:55.589 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000180888 s, 2.8 MB/s 2024-03-04T12:10:55.589 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T12:10:55.644 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:55.644 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T12:10:55.645 INFO:tasks.ceph:Generating config... 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] debug client = 20 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] debug ms = 1 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] pid file = 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-04T12:10:55.646 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-04T12:10:55.646 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-04T12:10:55.646 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-04T12:10:55.646 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-04T12:10:55.646 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-04T12:10:55.647 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-04T12:10:55.647 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-04T12:10:55.647 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-04T12:10:55.647 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-04T12:10:55.648 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-04T12:10:55.648 INFO:tasks.ceph:Setting up mon.a... 2024-03-04T12:10:55.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-04T12:10:55.704 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.keyring 2024-03-04T12:10:55.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-04T12:10:55.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T12:10:55.819 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.148'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2024-03-04T12:10:55.820 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.148,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-04T12:10:55.821 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:55.821 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T12:10:55.875 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.148 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:10:55.959 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: generated fsid d4b3ed31-a94c-42d0-968e-1d6a4e199a28 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = quincy 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 0 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:fsid d4b3ed31-a94c-42d0-968e-1d6a4e199a28 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:last_changed 2024-03-04T12:10:55.960578+0000 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:created 2024-03-04T12:10:55.960578+0000 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:min_mon_release 17 (quincy) 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:election_strategy: 1 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:0: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.a 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2024-03-04T12:10:55.960 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-04T12:10:55.961 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T12:10:55.976 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d4b3ed31-a94c-42d0-968e-1d6a4e199a28... 2024-03-04T12:10:55.977 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T12:10:56.018 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T12:10:56.066 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-03-04T12:10:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-03-04T12:10:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T12:10:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-03-04T12:10:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T12:10:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T12:10:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = d4b3ed31-a94c-42d0-968e-1d6a4e199a28 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.069 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-03-04T12:10:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-03-04T12:10:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-03-04T12:10:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T12:10:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-03-04T12:10:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-03-04T12:10:56.076 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-04T12:10:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-03-04T12:10:56.078 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2024-03-04T12:10:56.079 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.080 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T12:10:56.081 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.148,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on skipped message = False 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = d4b3ed31-a94c-42d0-968e-1d6a4e199a28 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.082 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2024-03-04T12:10:56.083 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug trim objects = true 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2024-03-04T12:10:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 20 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 20 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2024-03-04T12:10:56.085 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 10 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-03-04T12:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-03-04T12:10:56.087 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.088 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2024-03-04T12:10:56.089 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-03-04T12:10:56.090 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds balancer = 20 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2024-03-04T12:10:56.091 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2024-03-04T12:10:56.093 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-04T12:10:56.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-04T12:10:56.190 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-04T12:10:56.190 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:56.190 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-04T12:10:56.204 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:56.205 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-04T12:10:56.260 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:10:56.260 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:56.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T12:10:56.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T12:10:56.339 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:10:56.339 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:10:56.394 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:10:56.394 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:56.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T12:10:56.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T12:10:56.433 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:10:56.434 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:10:56.488 INFO:tasks.ceph:Setting up mon nodes... 2024-03-04T12:10:56.488 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-04T12:10:56.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T12:10:56.542 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T12:10:56.546 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T12:10:56.618 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T12:10:56.620 INFO:tasks.ceph:Setting up mds nodes... 2024-03-04T12:10:56.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-04T12:10:56.676 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-04T12:10:56.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-04T12:10:56.705 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-04T12:10:56.800 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-04T12:10:56.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-04T12:10:56.830 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-04T12:10:56.885 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-04T12:10:56.887 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-04T12:10:56.914 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-04T12:10:57.008 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-04T12:10:57.011 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-04T12:10:57.038 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-04T12:10:57.038 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T12:10:57.081 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-04T12:10:57.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-04T12:10:57.178 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-04T12:10:57.193 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-04T12:10:57.235 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-04T12:10:57.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-04T12:10:57.332 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-04T12:10:57.347 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-04T12:10:57.347 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T12:10:57.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-04T12:10:57.375 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:57.375 INFO:tasks.ceph:role: osd.0 2024-03-04T12:10:57.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:10:57.375 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:10:57.448 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:10:57.449 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:10:57.449 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:10:57.967 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-04T12:10:57.970 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-04T12:10:57.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-04T12:10:58.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-04T12:10:58.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-04T12:10:58.198 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:58.198 INFO:tasks.ceph:role: osd.1 2024-03-04T12:10:58.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:10:58.198 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T12:10:58.272 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:10:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:10:58.716 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-04T12:10:58.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-04T12:10:58.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-04T12:10:58.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-04T12:10:58.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-04T12:10:58.910 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:58.910 INFO:tasks.ceph:role: osd.2 2024-03-04T12:10:58.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:10:58.910 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T12:10:58.984 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:10:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:10:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:10:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:10:58.984 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:10:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:10:58.985 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:10:58.985 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:10:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:10:58.985 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:10:59.469 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-04T12:10:59.471 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-04T12:10:59.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-04T12:10:59.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-04T12:10:59.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-04T12:10:59.721 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T12:10:59.721 INFO:tasks.ceph:role: osd.3 2024-03-04T12:10:59.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:10:59.721 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:10:59.796 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:10:59.797 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:10:59.797 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:11:00.306 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-04T12:11:00.308 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-04T12:11:00.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-04T12:11:00.456 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-04T12:11:00.489 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.590+0000 7fe0e180f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.590+0000 7fe0e180f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.590+0000 7fe0e180f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.590+0000 7fe0e180f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.590+0000 7fe0e180f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T12:11:00.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:00.591+0000 7fe0e180f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-04T12:11:03.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T12:11:03.480 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.572+0000 7fe2fbe01640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-04T12:11:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.572+0000 7fe2fbe01640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-04T12:11:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.572+0000 7fe2fbe01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T12:11:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.572+0000 7fe2fbe01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T12:11:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.572+0000 7fe2fbe01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T12:11:03.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:03.573+0000 7fe2fbe01640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-04T12:11:06.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T12:11:06.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:06.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.543+0000 7fba8e28b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-04T12:11:06.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.543+0000 7fba8e28b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-04T12:11:06.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.543+0000 7fba8e28b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T12:11:06.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.543+0000 7fba8e28b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T12:11:06.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.543+0000 7fba8e28b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T12:11:06.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:06.544+0000 7fba8e28b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-04T12:11:09.394 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T12:11:09.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.516+0000 7fbbd75f8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-04T12:11:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.516+0000 7fbbd75f8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-04T12:11:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.516+0000 7fbbd75f8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T12:11:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.516+0000 7fbbd75f8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T12:11:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.516+0000 7fbbd75f8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T12:11:09.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:09.517+0000 7fbbd75f8640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-04T12:11:12.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T12:11:12.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-04T12:11:12.427 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T12:11:12.427 INFO:tasks.ceph:role: osd.4 2024-03-04T12:11:12.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:11:12.428 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:11:12.506 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:11:12.999 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-04T12:11:13.002 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-04T12:11:13.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-04T12:11:13.151 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-04T12:11:13.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-04T12:11:13.253 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T12:11:13.253 INFO:tasks.ceph:role: osd.5 2024-03-04T12:11:13.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:11:13.253 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T12:11:13.328 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:11:13.328 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:11:13.328 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:11:13.328 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:11:13.328 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:11:13.329 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:11:13.329 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:11:13.329 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:11:13.329 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:11:13.329 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:11:13.774 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-04T12:11:13.776 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-04T12:11:13.777 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-04T12:11:13.859 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-04T12:11:13.930 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-04T12:11:13.997 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T12:11:13.997 INFO:tasks.ceph:role: osd.6 2024-03-04T12:11:13.997 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:11:13.997 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:11:14.072 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:11:14.073 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:11:14.073 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:11:14.558 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-04T12:11:14.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-04T12:11:14.560 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-04T12:11:14.702 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-04T12:11:14.736 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-04T12:11:14.804 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T12:11:14.804 INFO:tasks.ceph:role: osd.7 2024-03-04T12:11:14.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:11:14.804 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T12:11:14.878 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T12:11:14.878 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T12:11:14.878 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T12:11:14.879 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T12:11:15.394 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-04T12:11:15.396 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-04T12:11:15.396 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-04T12:11:15.543 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-04T12:11:15.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.677+0000 7fdf8911f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.677+0000 7fdf8911f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.677+0000 7fdf8911f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.678+0000 7fdf8911f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.678+0000 7fdf8911f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T12:11:15.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:15.678+0000 7fdf8911f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-04T12:11:18.533 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T12:11:18.564 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:18.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.654+0000 7fe7894f9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-04T12:11:18.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.654+0000 7fe7894f9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-04T12:11:18.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.654+0000 7fe7894f9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T12:11:18.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.654+0000 7fe7894f9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T12:11:18.655 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.654+0000 7fe7894f9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T12:11:18.656 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:18.655+0000 7fe7894f9640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-04T12:11:21.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T12:11:21.544 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:21.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-04T12:11:21.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-04T12:11:21.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T12:11:21.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T12:11:21.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T12:11:21.636 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:21.635+0000 7ff15291b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-04T12:11:24.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:11:24.069 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:11:24.487 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-04T12:11:24.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.603+0000 7f0eb460c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-04T12:11:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.603+0000 7f0eb460c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-04T12:11:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.603+0000 7f0eb460c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T12:11:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.603+0000 7f0eb460c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T12:11:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.603+0000 7f0eb460c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T12:11:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:24.604+0000 7f0eb460c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-04T12:11:27.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-04T12:11:27.479 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-04T12:11:27.479 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.479 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-04T12:11:27.511 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.511 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-04T12:11:27.584 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-04T12:11:27.654 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-04T12:11:27.723 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.723 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-04T12:11:27.793 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-04T12:11:27.861 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:27.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-04T12:11:27.930 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:27.930 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-04T12:11:27.962 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:27.962 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-04T12:11:28.031 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-04T12:11:28.100 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-04T12:11:28.169 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.169 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-04T12:11:28.238 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-04T12:11:28.306 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.307 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-04T12:11:28.374 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:28.374 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-04T12:11:28.391 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:28.392 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-04T12:11:28.449 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.449 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-04T12:11:28.463 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:28.463 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-04T12:11:28.518 INFO:tasks.ceph:Adding keys to all mons... 2024-03-04T12:11:28.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T12:11:28.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T12:11:28.561 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-03-04T12:11:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVlxdJmIBAA2Dj8FYvTvXbue2805GEJQw== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVlPTpZKBAAPdky8L7gzakTpgf1zEyLyw== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVl65bELxAATNugvAPFPxY3GrpmmdX9Lg== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBUuuVlvsE+IxAAQjpgDYDw3OZLznNBrBlUyA== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBXuuVliEcuIhAA9hXBDscdE49SjCXZUT9n6w== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBauuVlMjJtIBAAnItsRXnawrqBExdXS3OHNQ== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBduuVly8HZHhAA+QcJYwEKRsx7gI7WFiEwIQ== 2024-03-04T12:11:28.562 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVl91znJBAAeC7+KjTaCJRMZzHjK2WifA== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVlxXXRNBAA6K+J/CFuWZuA54WYAwv1OA== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlrOGOABAACt3dfy2qT01wPlK7Kp+bPw== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBjuuVlFwF1KBAAIReAK44KV03xz6DAEsgQ7Q== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBmuuVlHTcRJxAAywci5A78GNGEDTDyVGAwBg== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBpuuVlopHnJRAAi4cCZCxM9p1Z2x9jaEn9Fw== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBsuuVlzF70IxAAxKllVg9IV4JxzNo8P8jHYw== 2024-03-04T12:11:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlHlfkBBAAcGWanX3ahyQy3t9Q4LBHWQ== 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlkW20ChAAg5QRFdWkrTWNys1Jd8i98g== 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlrNcUDhAALPWdoKn42fRfmcZKUB6/iA== 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2024-03-04T12:11:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlBIrYExAAS6Y8zD2oL7igsrheXxgZoA== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBQuuVlxdJmIBAA2Dj8FYvTvXbue2805GEJQw== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBQuuVlPTpZKBAAPdky8L7gzakTpgf1zEyLyw== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBQuuVl65bELxAATNugvAPFPxY3GrpmmdX9Lg== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBUuuVlvsE+IxAAQjpgDYDw3OZLznNBrBlUyA== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBXuuVliEcuIhAA9hXBDscdE49SjCXZUT9n6w== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBauuVlMjJtIBAAnItsRXnawrqBExdXS3OHNQ== 2024-03-04T12:11:28.584 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBduuVly8HZHhAA+QcJYwEKRsx7gI7WFiEwIQ== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBQuuVl91znJBAAeC7+KjTaCJRMZzHjK2WifA== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBQuuVlxXXRNBAA6K+J/CFuWZuA54WYAwv1OA== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBRuuVlrOGOABAACt3dfy2qT01wPlK7Kp+bPw== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBjuuVlFwF1KBAAIReAK44KV03xz6DAEsgQ7Q== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBmuuVlHTcRJxAAywci5A78GNGEDTDyVGAwBg== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBpuuVlopHnJRAAi4cCZCxM9p1Z2x9jaEn9Fw== 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2024-03-04T12:11:28.585 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBsuuVlzF70IxAAxKllVg9IV4JxzNo8P8jHYw== 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBRuuVlHlfkBBAAcGWanX3ahyQy3t9Q4LBHWQ== 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBRuuVlkW20ChAAg5QRFdWkrTWNys1Jd8i98g== 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBRuuVlrNcUDhAALPWdoKn42fRfmcZKUB6/iA== 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2024-03-04T12:11:28.586 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBRuuVlBIrYExAAS6Y8zD2oL7igsrheXxgZoA== 2024-03-04T12:11:28.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T12:11:28.608 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T12:11:28.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:28.681 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:28.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:28.739 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:28.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.796 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.851 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:28.959 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T12:11:29.012 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T12:11:29.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:29.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:29.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:29.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T12:11:29.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.179 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.286 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.340 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T12:11:29.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.565 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T12:11:29.619 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-04T12:11:29.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-04T12:11:29.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T12:11:29.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-04T12:11:29.769 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-04T12:11:29.795 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T12:11:29.892 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-04T12:11:29.919 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-04T12:11:29.989 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T12:11:30.086 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-04T12:11:30.113 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:30.115 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T12:11:30.167 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-04T12:11:30.167 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-04T12:11:30.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-04T12:11:30.172 INFO:tasks.ceph.mon.a:Started 2024-03-04T12:11:30.172 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-04T12:11:30.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-04T12:11:30.210 INFO:tasks.ceph.mon.b:Started 2024-03-04T12:11:30.210 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-04T12:11:30.210 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-04T12:11:30.212 INFO:tasks.ceph.mon.c:Started 2024-03-04T12:11:30.212 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-04T12:11:30.212 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-04T12:11:30.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-04T12:11:30.214 INFO:tasks.ceph.mgr.y:Started 2024-03-04T12:11:30.215 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-04T12:11:30.215 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-04T12:11:30.216 INFO:tasks.ceph.mgr.x:Started 2024-03-04T12:11:30.217 INFO:tasks.ceph:Setting crush tunables to default 2024-03-04T12:11:30.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd crush tunables default 2024-03-04T12:11:30.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.319+0000 7f8951905640 1 Processor -- start 2024-03-04T12:11:30.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.319+0000 7f8951905640 1 -- start start 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8951905640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8951905640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8951905640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c0d3510 con 0x7f894c15dc60 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8951905640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c162720 con 0x7f894c0b9420 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8951905640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0d19f0 con 0x7f894c0b5c90 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:58770/0 (socket says 172.21.15.148:58770) 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894b7fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 -- 172.21.15.148:0/1229396770 learned_addr learned my addr 172.21.15.148:0/1229396770 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894affd640 1 --1- 172.21.15.148:0/1229396770 >> v1:172.21.15.191:6789/0 conn(0x7f894c15dc60 0x7f894c0b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:41820/0 (socket says 172.21.15.148:41820) 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f894c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-04T12:11:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f894c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 msgr2=0x7f894c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 msgr2=0x7f894c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T12:11:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.320+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f8943fff640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f894c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f8943fff640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f894c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 msgr2=0x7f894c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-04T12:11:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 msgr2=0x7f894c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T12:11:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T12:11:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T12:11:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.521+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342083356 0 0) 0x7f894c0d3510 con 0x7f894c15dc60 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8938003520 con 0x7f894c15dc60 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4202216311 0 0) 0x7f8938003520 con 0x7f894c15dc60 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f894c1621e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f894c1621e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 msgr2=0x7f894c0ce150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 --2- 172.21.15.148:0/1229396770 >> v2:172.21.15.191:3300/0 conn(0x7f894c0b9420 0x7f894c0ce150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c0b9920 con 0x7f894c15dc60 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f893c001670 con 0x7f894c15dc60 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 >> v1:172.21.15.191:6789/0 conn(0x7f894c15dc60 legacy=0x7f894c0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f89380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938003fe0 0x7f8938004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8938004d40 0x7f89380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c162720 con 0x7f8938003fe0 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0d19f0 con 0x7f8938004d40 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893800bac0 con 0x7f894c0b5c90 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8938004d40 0x7f89380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938003fe0 0x7f8938004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.528+0000 7f894affd640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f89380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 msgr2=0x7f89380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c0b5c90 0x7f89380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938003fe0 msgr2=0x7f8938004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938003fe0 0x7f8938004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938003520 con 0x7f8938004d40 2024-03-04T12:11:30.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.529+0000 7f8943fff640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938003fe0 0x7f8938004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:30.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.530+0000 7f894b7fe640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8938004d40 0x7f89380051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f894c0b9850 tx=0x7f8938005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dee7a404f7f0a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.530+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944038070 con 0x7f8938004d40 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.530+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944033600 con 0x7f8938004d40 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.530+0000 7f894a7fc640 1 -- 172.21.15.148:0/1229396770 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944004040 con 0x7f8938004d40 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 -- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8938004d40 msgr2=0x7f89380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 --2- 172.21.15.148:0/1229396770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8938004d40 0x7f89380051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f894c0b9850 tx=0x7f8938005830 comp rx=0 tx=0).stop 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f894b7fe640 1 -- 172.21.15.148:0/1229396770 reap_dead start 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 -- 172.21.15.148:0/1229396770 shutdown_connections 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 -- 172.21.15.148:0/1229396770 >> 172.21.15.148:0/1229396770 conn(0x7f894c0d0580 msgr2=0x7f894c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 -- 172.21.15.148:0/1229396770 shutdown_connections 2024-03-04T12:11:30.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.531+0000 7f8951905640 1 -- 172.21.15.148:0/1229396770 wait complete. 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.532+0000 7f8951905640 1 Processor -- start 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.532+0000 7f8951905640 1 -- start start 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 0x7f894c1f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f894affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f894affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49772/0 (socket says 172.21.15.148:49772) 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c153d90 0x7f894c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:30.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0d1f20 con 0x7f8938004d40 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f894affd640 1 -- 172.21.15.148:0/3652730667 learned_addr learned my addr 172.21.15.148:0/3652730667 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0d2290 con 0x7f894c155330 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c15e040 con 0x7f894c153d90 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.533+0000 7f8943fff640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 0x7f894c1f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894b7fe640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c153d90 0x7f894c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 -- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c153d90 msgr2=0x7f894c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c153d90 0x7f894c15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 -- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 msgr2=0x7f894c1f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 0x7f894c1f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c154290 con 0x7f8938004d40 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f8943fff640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 0x7f894c1f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:30.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.534+0000 7f894affd640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f893c015fd0 tx=0x7f893c007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:30.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c01f420 con 0x7f8938004d40 2024-03-04T12:11:30.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c01f600 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c154570 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c01f9f0 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c154b30 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f893c0297d0 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.535+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f893c028db0 con 0x7f8938004d40 2024-03-04T12:11:30.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.536+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c074f60 con 0x7f8938004d40 2024-03-04T12:11:30.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.538+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f893c024030 con 0x7f8938004d40 2024-03-04T12:11:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.604+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f894c0687d0 con 0x7f8938004d40 2024-03-04T12:11:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.606+0000 7f8948ff9640 1 -- 172.21.15.148:0/3652730667 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f893c0283e0 con 0x7f8938004d40 2024-03-04T12:11:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 msgr2=0x7f894c1f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f893c015fd0 tx=0x7f893c007660 comp rx=0 tx=0).stop 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 shutdown_connections 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f894c153d90 0x7f894c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f894c155330 0x7f894c1f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 --2- 172.21.15.148:0/3652730667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8938004d40 0x7f894c1f3ca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 >> 172.21.15.148:0/3652730667 conn(0x7f894c0d0580 msgr2=0x7f894c0b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 shutdown_connections 2024-03-04T12:11:30.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:30.607+0000 7f8951905640 1 -- 172.21.15.148:0/3652730667 wait complete. 2024-03-04T12:11:30.619 INFO:tasks.ceph:check_enable_crimson: False 2024-03-04T12:11:30.619 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-04T12:11:30.619 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:30.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-04T12:11:30.646 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:30.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-04T12:11:30.713 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:30.714 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-04T12:11:30.780 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:11:30.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-04T12:11:30.848 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:30.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-04T12:11:30.878 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:30.878 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-04T12:11:30.947 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:30.948 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-04T12:11:31.011 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.011+0000 7fc40981d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T12:11:31.012 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:11:31.013 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-04T12:11:31.025 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.024+0000 7fa5ebe39040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T12:11:31.067 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.066+0000 7fc40981d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T12:11:31.079 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 76b5cd59-dfd9-4906-8f61-b9057971ed30 0 2024-03-04T12:11:31.080 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.078+0000 7fa5ebe39040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T12:11:31.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.219+0000 7fba61859640 1 Processor -- start 2024-03-04T12:11:31.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 -- start start 2024-03-04T12:11:31.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba5c0d34a0 0x7fba5c06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c0d2400 con 0x7fba5c0b9410 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0d26e0 con 0x7fba5c0d34a0 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba61859640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0d29c0 con 0x7fba5c0b5700 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.220+0000 7fba5affd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46856/0 (socket says 172.21.15.191:46856) 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 -- 172.21.15.191:0/3560784161 learned_addr learned my addr 172.21.15.191:0/3560784161 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5a7fc640 1 --2- 172.21.15.191:0/3560784161 >> v2:172.21.15.148:3300/0 conn(0x7fba5c0d34a0 0x7fba5c06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 -- 172.21.15.191:0/3560784161 >> v2:172.21.15.148:3300/0 conn(0x7fba5c0d34a0 msgr2=0x7fba5c06c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 --2- 172.21.15.191:0/3560784161 >> v2:172.21.15.148:3300/0 conn(0x7fba5c0d34a0 0x7fba5c06c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 -- 172.21.15.191:0/3560784161 >> v1:172.21.15.148:6789/0 conn(0x7fba5c0b9410 legacy=0x7fba5c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 -- 172.21.15.191:0/3560784161 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0d2ca0 con 0x7fba5c0b5700 2024-03-04T12:11:31.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.221+0000 7fba5affd640 1 --2- 172.21.15.191:0/3560784161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba50009870 tx=0x7fba5002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240128367fea2b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba59ffb640 1 -- 172.21.15.191:0/3560784161 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50035020 con 0x7fba5c0b5700 2024-03-04T12:11:31.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba59ffb640 1 -- 172.21.15.191:0/3560784161 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50031640 con 0x7fba5c0b5700 2024-03-04T12:11:31.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba59ffb640 1 -- 172.21.15.191:0/3560784161 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5002d040 con 0x7fba5c0b5700 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 -- 172.21.15.191:0/3560784161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 msgr2=0x7fba5c0d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 --2- 172.21.15.191:0/3560784161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba50009870 tx=0x7fba5002b8f0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 -- 172.21.15.191:0/3560784161 shutdown_connections 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 --2- 172.21.15.191:0/3560784161 >> v2:172.21.15.148:3300/0 conn(0x7fba5c0d34a0 0x7fba5c06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 --2- 172.21.15.191:0/3560784161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0b5700 0x7fba5c0d1b00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 -- 172.21.15.191:0/3560784161 >> 172.21.15.191:0/3560784161 conn(0x7fba5c0d0490 msgr2=0x7fba5c0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 -- 172.21.15.191:0/3560784161 shutdown_connections 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.222+0000 7fba61859640 1 -- 172.21.15.191:0/3560784161 wait complete. 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 Processor -- start 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 -- start start 2024-03-04T12:11:31.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 0x7fba5c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 0x7fba5c145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c06ceb0 con 0x7fba5c0b5700 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c06cd30 con 0x7fba5c0b9410 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.224+0000 7fba61859640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0b5ae0 con 0x7fba5c0d34a0 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 0x7fba5c145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 0x7fba5c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.191:36812/0 (socket says 172.21.15.191:36812) 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5affd640 1 -- 172.21.15.191:0/4091592512 learned_addr learned my addr 172.21.15.191:0/4091592512 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 0x7fba5c145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46862/0 (socket says 172.21.15.191:46862) 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 -- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 msgr2=0x7fba5c145840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 0x7fba5c145840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.228 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 -- 172.21.15.191:0/4091592512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 msgr2=0x7fba5c1437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 -- 172.21.15.191:0/4091592512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c153780 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5affd640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba5a7fc640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 0x7fba5c0c9ca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e870 tx=0x7fba4c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c017070 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00ee60 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c153a60 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00cb60 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c0cdaa0 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fba4c01a070 con 0x7fba5c0b9410 2024-03-04T12:11:31.229 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.225+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba4c01c790 con 0x7fba5c0b9410 2024-03-04T12:11:31.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.227+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c074f60 con 0x7fba5c0b9410 2024-03-04T12:11:31.230 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.229+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fba4c00cd00 con 0x7fba5c0b9410 2024-03-04T12:11:31.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.299+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "76b5cd59-dfd9-4906-8f61-b9057971ed30", "id": 0} v 0) v1 -- 0x7fba5c0ce3a0 con 0x7fba5c0b9410 2024-03-04T12:11:31.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.302+0000 7fba43fff640 1 -- 172.21.15.191:0/4091592512 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76b5cd59-dfd9-4906-8f61-b9057971ed30", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba4c01cab0 con 0x7fba5c0b9410 2024-03-04T12:11:31.303 INFO:teuthology.orchestra.run.smithi191.stdout:0 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 msgr2=0x7fba5c0c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 0x7fba5c0c9ca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e870 tx=0x7fba4c00ed30 comp rx=0 tx=0).stop 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 shutdown_connections 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba5c0d34a0 0x7fba5c145840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba5c0b9410 0x7fba5c0c9ca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 --2- 172.21.15.191:0/4091592512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba5c0b5700 0x7fba5c1437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 >> 172.21.15.191:0/4091592512 conn(0x7fba5c0d0490 msgr2=0x7fba5c1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 shutdown_connections 2024-03-04T12:11:31.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.303+0000 7fba61859640 1 -- 172.21.15.191:0/4091592512 wait complete. 2024-03-04T12:11:31.314 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new af8b755c-212e-46c7-acac-dd927ca0da2e 1 2024-03-04T12:11:31.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.413+0000 7f8478129640 1 Processor -- start 2024-03-04T12:11:31.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.413+0000 7f8478129640 1 -- start start 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f8478129640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84700af420 0x7f84700cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f8478129640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f8478129640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700cead0 con 0x7f847015bba0 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f8478129640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700ced70 con 0x7f84700af420 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f8478129640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cf050 con 0x7f84700cc310 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46872/0 (socket says 172.21.15.191:46872) 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 -- 172.21.15.191:0/1115812635 learned_addr learned my addr 172.21.15.191:0/1115812635 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84756c4640 1 --2- 172.21.15.191:0/1115812635 >> v2:172.21.15.148:3300/0 conn(0x7f84700af420 0x7f84700cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 -- 172.21.15.191:0/1115812635 >> v2:172.21.15.148:3300/0 conn(0x7f84700af420 msgr2=0x7f84700cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:31.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 --2- 172.21.15.191:0/1115812635 >> v2:172.21.15.148:3300/0 conn(0x7f84700af420 0x7f84700cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 -- 172.21.15.191:0/1115812635 >> v1:172.21.15.148:6789/0 conn(0x7f847015bba0 legacy=0x7f84700cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.414+0000 7f84766c6640 1 -- 172.21.15.191:0/1115812635 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847006ad70 con 0x7f84700cc310 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.415+0000 7f84766c6640 1 --2- 172.21.15.191:0/1115812635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e50 tx=0x7f846802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3698d721fe455121 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.415+0000 7f8474ec3640 1 -- 172.21.15.191:0/1115812635 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846802cb00 con 0x7f84700cc310 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.415+0000 7f8474ec3640 1 -- 172.21.15.191:0/1115812635 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846802cca0 con 0x7f84700cc310 2024-03-04T12:11:31.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.415+0000 7f8474ec3640 1 -- 172.21.15.191:0/1115812635 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468034610 con 0x7f84700cc310 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 -- 172.21.15.191:0/1115812635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 msgr2=0x7f847006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 --2- 172.21.15.191:0/1115812635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e50 tx=0x7f846802bf60 comp rx=0 tx=0).stop 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 -- 172.21.15.191:0/1115812635 shutdown_connections 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 --2- 172.21.15.191:0/1115812635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700cc310 0x7f847006b570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 --2- 172.21.15.191:0/1115812635 >> v2:172.21.15.148:3300/0 conn(0x7f84700af420 0x7f84700cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 -- 172.21.15.191:0/1115812635 >> 172.21.15.191:0/1115812635 conn(0x7f84700cc970 msgr2=0x7f84700ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 -- 172.21.15.191:0/1115812635 shutdown_connections 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.416+0000 7f8478129640 1 -- 172.21.15.191:0/1115812635 wait complete. 2024-03-04T12:11:31.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.417+0000 7f8478129640 1 Processor -- start 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.417+0000 7f8478129640 1 -- start start 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.417+0000 7f8478129640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84700cc310 0x7f84700b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 0x7f84700c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847009e2d0 con 0x7f84700cc310 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470066be0 con 0x7f847015bba0 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847006bab0 con 0x7f84700af420 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84766c6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 0x7f84700c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46880/0 (socket says 172.21.15.191:46880) 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 -- 172.21.15.191:0/1765102060 learned_addr learned my addr 172.21.15.191:0/1765102060 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84756c4640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84700cc310 0x7f84700b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 -- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 msgr2=0x7f84700c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 0x7f84700c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 -- 172.21.15.191:0/1765102060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84700cc310 msgr2=0x7f84700b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84700cc310 0x7f84700b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 -- 172.21.15.191:0/1765102060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847006df60 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84766c6640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 0x7f84700c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8475ec5640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c3f0 tx=0x7f846c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0052d0 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c005490 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847015e720 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c007260 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847015e400 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f846c007400 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.418+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f846c003ad0 con 0x7f84700af420 2024-03-04T12:11:31.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.419+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470074f60 con 0x7f84700af420 2024-03-04T12:11:31.423 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.421+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f846c003d20 con 0x7f84700af420 2024-03-04T12:11:31.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.495+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "af8b755c-212e-46c7-acac-dd927ca0da2e", "id": 1} v 0) v1 -- 0x7f84700b0910 con 0x7f84700af420 2024-03-04T12:11:31.501 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.500+0000 7f84667fc640 1 -- 172.21.15.191:0/1765102060 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af8b755c-212e-46c7-acac-dd927ca0da2e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f846c0035f0 con 0x7f84700af420 2024-03-04T12:11:31.502 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 msgr2=0x7f847015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c3f0 tx=0x7f846c00c8b0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 shutdown_connections 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f847015bba0 0x7f84700c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84700cc310 0x7f84700b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 --2- 172.21.15.191:0/1765102060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84700af420 0x7f847015dec0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 >> 172.21.15.191:0/1765102060 conn(0x7f84700cc970 msgr2=0x7f84700b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.501+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 shutdown_connections 2024-03-04T12:11:31.503 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.502+0000 7f8478129640 1 -- 172.21.15.191:0/1765102060 wait complete. 2024-03-04T12:11:31.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new e967f2b6-df78-4a51-af8c-b1e26fe8b4d0 2 2024-03-04T12:11:31.601 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.600+0000 7fa5ebe39040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T12:11:31.612 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.611+0000 7fc40981d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T12:11:31.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.612+0000 7feff42c0640 1 Processor -- start 2024-03-04T12:11:31.613 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.612+0000 7feff42c0640 1 -- start start 2024-03-04T12:11:31.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff42c0640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fefec06c060 0x7fefec0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.614 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff42c0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0cf740 con 0x7fefec161390 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff42c0640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec06c7d0 con 0x7fefec0cec20 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff42c0640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec06d3e0 con 0x7fefec06c060 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fefec06c060 0x7fefec0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fefec06c060 0x7fefec0ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55360/0 (socket says 172.21.15.191:55360) 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 -- 172.21.15.191:0/2768881802 learned_addr learned my addr 172.21.15.191:0/2768881802 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 -- 172.21.15.191:0/2768881802 >> v1:172.21.15.191:6789/0 conn(0x7fefec0cec20 legacy=0x7fefec0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 -- 172.21.15.191:0/2768881802 >> v1:172.21.15.148:6789/0 conn(0x7fefec161390 legacy=0x7fefec0d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.613+0000 7feff185b640 1 -- 172.21.15.191:0/2768881802 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec06cab0 con 0x7fefec06c060 2024-03-04T12:11:31.616 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.614+0000 7feff185b640 1 --2- 172.21.15.191:0/2768881802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec06c060 0x7fefec0ce6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009870 tx=0x7fefe802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dcfca7eb2b0486a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.614+0000 7feff105a640 1 -- 172.21.15.191:0/2768881802 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe802f840 con 0x7fefec06c060 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.614+0000 7feff105a640 1 -- 172.21.15.191:0/2768881802 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe802f9e0 con 0x7fefec06c060 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.614+0000 7feff105a640 1 -- 172.21.15.191:0/2768881802 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe802fdb0 con 0x7fefec06c060 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 -- 172.21.15.191:0/2768881802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec06c060 msgr2=0x7fefec0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 --2- 172.21.15.191:0/2768881802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec06c060 0x7fefec0ce6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009870 tx=0x7fefe802ba20 comp rx=0 tx=0).stop 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 -- 172.21.15.191:0/2768881802 shutdown_connections 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 --2- 172.21.15.191:0/2768881802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec06c060 0x7fefec0ce6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 -- 172.21.15.191:0/2768881802 >> 172.21.15.191:0/2768881802 conn(0x7fefec066dc0 msgr2=0x7fefec0671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 -- 172.21.15.191:0/2768881802 shutdown_connections 2024-03-04T12:11:31.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.615+0000 7feff42c0640 1 -- 172.21.15.191:0/2768881802 wait complete. 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.616+0000 7feff42c0640 1 Processor -- start 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.616+0000 7feff42c0640 1 -- start start 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefec0cec20 0x7fefec1e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06c4c0 con 0x7fefec0cec20 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec06d070 con 0x7fefec161390 2024-03-04T12:11:31.620 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff42c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec161770 con 0x7fefec06c060 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff205c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff285d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff205c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46892/0 (socket says 172.21.15.191:46892) 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff205c640 1 -- 172.21.15.191:0/3883757807 learned_addr learned my addr 172.21.15.191:0/3883757807 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff285d640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55364/0 (socket says 172.21.15.191:55364) 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff185b640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefec0cec20 0x7fefec1e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.617+0000 7feff285d640 1 -- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 msgr2=0x7fefec14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff285d640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff285d640 1 -- 172.21.15.191:0/3883757807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefec0cec20 msgr2=0x7fefec1e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff285d640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefec0cec20 0x7fefec1e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff285d640 1 -- 172.21.15.191:0/3883757807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec15d0a0 con 0x7fefec161390 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff205c640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:31.621 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff285d640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c690 tx=0x7fefe400cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4003b70 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4003d10 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec1e7260 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4005310 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec1e7820 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fefe40054b0 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.618+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fefe4012890 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.619+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec1e7a60 con 0x7fefec161390 2024-03-04T12:11:31.622 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.621+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fefe4012470 con 0x7fefec161390 2024-03-04T12:11:31.656 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.655+0000 7fa5ebe39040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T12:11:31.669 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.668+0000 7fc40981d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T12:11:31.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.693+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e967f2b6-df78-4a51-af8c-b1e26fe8b4d0", "id": 2} v 0) v1 -- 0x7fefec0b77c0 con 0x7fefec161390 2024-03-04T12:11:31.697 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.696+0000 7fefe2ffd640 1 -- 172.21.15.191:0/3883757807 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e967f2b6-df78-4a51-af8c-b1e26fe8b4d0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fefe4012b70 con 0x7fefec161390 2024-03-04T12:11:31.698 INFO:teuthology.orchestra.run.smithi191.stdout:2 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 msgr2=0x7fefec1e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c690 tx=0x7fefe400cb50 comp rx=0 tx=0).stop 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 shutdown_connections 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefec161390 0x7fefec1e6c30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefec0cec20 0x7fefec1e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 --2- 172.21.15.191:0/3883757807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefec06c060 0x7fefec14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 >> 172.21.15.191:0/3883757807 conn(0x7fefec066dc0 msgr2=0x7fefec0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 shutdown_connections 2024-03-04T12:11:31.699 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.697+0000 7feff42c0640 1 -- 172.21.15.191:0/3883757807 wait complete. 2024-03-04T12:11:31.701 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.701+0000 7fa5ebe39040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T12:11:31.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 41ce11cf-66ba-44e8-a0e5-3e976505e466 3 2024-03-04T12:11:31.717 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.716+0000 7fc40981d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T12:11:31.755 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.755+0000 7fa5ebe39040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T12:11:31.775 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.774+0000 7fc40981d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T12:11:31.798 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.798+0000 7fa5ebe39040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T12:11:31.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.809+0000 7fd95c5b3640 1 Processor -- start 2024-03-04T12:11:31.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.809+0000 7fd95c5b3640 1 -- start start 2024-03-04T12:11:31.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95c5b3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd9540b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95c5b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9540ceec0 0x7fd9540b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95c5b3640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95415ca50 con 0x7fd95415c2e0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95c5b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95415d2f0 con 0x7fd9540ceec0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95c5b3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95415d6d0 con 0x7fd9540ccbd0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95a34f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd9540b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd959b4e640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fd95415c2e0 0x7fd9540ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35184/0 (socket says 172.21.15.191:35184) 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd959b4e640 1 -- 172.21.15.191:0/1245005632 learned_addr learned my addr 172.21.15.191:0/1245005632 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95ab50640 1 --2- 172.21.15.191:0/1245005632 >> v2:172.21.15.148:3300/0 conn(0x7fd9540ceec0 0x7fd9540b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95934d640 1 -- 172.21.15.191:0/1245005632 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160438344 0 0) 0x7fd95415ca50 con 0x7fd95415c2e0 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95a34f640 1 -- 172.21.15.191:0/1245005632 >> v2:172.21.15.148:3300/0 conn(0x7fd9540ceec0 msgr2=0x7fd9540b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95a34f640 1 --2- 172.21.15.191:0/1245005632 >> v2:172.21.15.148:3300/0 conn(0x7fd9540ceec0 0x7fd9540b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95a34f640 1 -- 172.21.15.191:0/1245005632 >> v1:172.21.15.191:6789/0 conn(0x7fd95415c2e0 legacy=0x7fd9540ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.810+0000 7fd95a34f640 1 -- 172.21.15.191:0/1245005632 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95415cd30 con 0x7fd9540ccbd0 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.811+0000 7fd95a34f640 1 --2- 172.21.15.191:0/1245005632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd9540b7320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd9440097a0 tx=0x7fd94402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ad5236232f1e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.811+0000 7fd95934d640 1 -- 172.21.15.191:0/1245005632 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944035020 con 0x7fd9540ccbd0 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.811+0000 7fd95934d640 1 -- 172.21.15.191:0/1245005632 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd944031810 con 0x7fd9540ccbd0 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.811+0000 7fd95934d640 1 -- 172.21.15.191:0/1245005632 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944031be0 con 0x7fd9540ccbd0 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 -- 172.21.15.191:0/1245005632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 msgr2=0x7fd9540b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/1245005632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd9540b7320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd9440097a0 tx=0x7fd94402b8b0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 -- 172.21.15.191:0/1245005632 shutdown_connections 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/1245005632 >> v2:172.21.15.148:3300/0 conn(0x7fd9540ceec0 0x7fd9540b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/1245005632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd9540b7320 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.814 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 -- 172.21.15.191:0/1245005632 >> 172.21.15.191:0/1245005632 conn(0x7fd954066dc0 msgr2=0x7fd9540671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 -- 172.21.15.191:0/1245005632 shutdown_connections 2024-03-04T12:11:31.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.812+0000 7fd95c5b3640 1 -- 172.21.15.191:0/1245005632 wait complete. 2024-03-04T12:11:31.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.813+0000 7fd95c5b3640 1 Processor -- start 2024-03-04T12:11:31.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.813+0000 7fd95c5b3640 1 -- start start 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9540ceec0 0x7fd9540b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95415c2e0 0x7fd9540b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9540ccfb0 con 0x7fd95415c2e0 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95415c760 con 0x7fd9540ceec0 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95c5b3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9540b0aa0 con 0x7fd9540ccbd0 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46918/0 (socket says 172.21.15.191:46918) 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd959b4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9540ceec0 0x7fd9540b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 -- 172.21.15.191:0/2509031928 learned_addr learned my addr 172.21.15.191:0/2509031928 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95ab50640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95415c2e0 0x7fd9540b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 -- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9540ceec0 msgr2=0x7fd9540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9540ceec0 0x7fd9540b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 -- 172.21.15.191:0/2509031928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95415c2e0 msgr2=0x7fd9540b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95415c2e0 0x7fd9540b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 -- 172.21.15.191:0/2509031928 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9540b0410 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.814+0000 7fd95a34f640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd94402bec0 tx=0x7fd9440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944041070 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94403ad00 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954156920 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944035040 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95406c160 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd94404c060 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.815+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd94403bc90 con 0x7fd9540ccbd0 2024-03-04T12:11:31.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.816+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd954074f60 con 0x7fd9540ccbd0 2024-03-04T12:11:31.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.818+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd944053020 con 0x7fd9540ccbd0 2024-03-04T12:11:31.821 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.820+0000 7fc40981d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T12:11:31.891 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.891+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "41ce11cf-66ba-44e8-a0e5-3e976505e466", "id": 3} v 0) v1 -- 0x7fd9540b0880 con 0x7fd9540ccbd0 2024-03-04T12:11:31.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.894+0000 7fd94affd640 1 -- 172.21.15.191:0/2509031928 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41ce11cf-66ba-44e8-a0e5-3e976505e466", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd94403b650 con 0x7fd9540ccbd0 2024-03-04T12:11:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:3 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 msgr2=0x7fd954154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd94402bec0 tx=0x7fd9440046c0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 shutdown_connections 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95415c2e0 0x7fd9540b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9540ceec0 0x7fd9540b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 --2- 172.21.15.191:0/2509031928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9540ccbd0 0x7fd954154e10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 >> 172.21.15.191:0/2509031928 conn(0x7fd954066dc0 msgr2=0x7fd9540cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:31.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 shutdown_connections 2024-03-04T12:11:31.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:31.895+0000 7fd95c5b3640 1 -- 172.21.15.191:0/2509031928 wait complete. 2024-03-04T12:11:31.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 3c655fef-c6c1-4080-8af7-217ce635c225 4 2024-03-04T12:11:31.915 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.915+0000 7fa5ebe39040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T12:11:31.941 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:31.940+0000 7fc40981d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T12:11:31.985 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:31.984+0000 7fa5ebe39040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T12:11:32.014 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.014+0000 7fc40981d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T12:11:32.031 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.030+0000 7fa5ebe39040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T12:11:32.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.040+0000 7f13f3116640 1 Processor -- start 2024-03-04T12:11:32.041 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.040+0000 7f13f3116640 1 -- start start 2024-03-04T12:11:32.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13f3116640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f13ec06c060 0x7f13ec0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.042 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13f3116640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0cf740 con 0x7f13ec0cec20 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13f3116640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec06c7d0 con 0x7f13ec161390 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13f3116640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec06d3e0 con 0x7f13ec06c060 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13ebfff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f13ec06c060 0x7f13ec0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13ebfff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f13ec06c060 0x7f13ec0ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55378/0 (socket says 172.21.15.191:55378) 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.041+0000 7f13ebfff640 1 -- 172.21.15.191:0/386193824 learned_addr learned my addr 172.21.15.191:0/386193824 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.042+0000 7f13ebfff640 1 -- 172.21.15.191:0/386193824 >> v1:172.21.15.191:6789/0 conn(0x7f13ec161390 legacy=0x7f13ec0d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.042+0000 7f13ebfff640 1 -- 172.21.15.191:0/386193824 >> v1:172.21.15.148:6789/0 conn(0x7f13ec0cec20 legacy=0x7f13ec0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.043 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.042+0000 7f13ebfff640 1 -- 172.21.15.191:0/386193824 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec06cab0 con 0x7f13ec06c060 2024-03-04T12:11:32.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.042+0000 7f13ebfff640 1 --2- 172.21.15.191:0/386193824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec06c060 0x7f13ec0ce6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009870 tx=0x7f13d802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9074d05d8865f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.042+0000 7f13eb7fe640 1 -- 172.21.15.191:0/386193824 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d802f840 con 0x7f13ec06c060 2024-03-04T12:11:32.044 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13eb7fe640 1 -- 172.21.15.191:0/386193824 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d802f9e0 con 0x7f13ec06c060 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13eb7fe640 1 -- 172.21.15.191:0/386193824 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d802fdb0 con 0x7f13ec06c060 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 -- 172.21.15.191:0/386193824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec06c060 msgr2=0x7f13ec0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 --2- 172.21.15.191:0/386193824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec06c060 0x7f13ec0ce6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009870 tx=0x7f13d802ba20 comp rx=0 tx=0).stop 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 -- 172.21.15.191:0/386193824 shutdown_connections 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 --2- 172.21.15.191:0/386193824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec06c060 0x7f13ec0ce6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 -- 172.21.15.191:0/386193824 >> 172.21.15.191:0/386193824 conn(0x7f13ec066dc0 msgr2=0x7f13ec0671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 -- 172.21.15.191:0/386193824 shutdown_connections 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.043+0000 7f13f3116640 1 -- 172.21.15.191:0/386193824 wait complete. 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.044+0000 7f13f3116640 1 Processor -- start 2024-03-04T12:11:32.045 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.044+0000 7f13f3116640 1 -- start start 2024-03-04T12:11:32.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec06c060 0x7f13ec0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13ec161390 0x7f13ec0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.047 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec06c4c0 con 0x7f13ec06c060 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec06d070 con 0x7f13ec0cec20 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f3116640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec161770 con 0x7f13ec161390 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13ebfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13ebfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55384/0 (socket says 172.21.15.191:55384) 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13ebfff640 1 -- 172.21.15.191:0/116576818 learned_addr learned my addr 172.21.15.191:0/116576818 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f16b3640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13ec161390 0x7f13ec0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.045+0000 7f13f0eb2640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec06c060 0x7f13ec0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 -- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13ec161390 msgr2=0x7f13ec0ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13ec161390 0x7f13ec0ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 -- 172.21.15.191:0/116576818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec06c060 msgr2=0x7f13ec0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec06c060 0x7f13ec0b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 -- 172.21.15.191:0/116576818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0b54e0 con 0x7f13ec0cec20 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13ebfff640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8041400 con 0x7f13ec0cec20 2024-03-04T12:11:32.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8046070 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8041800 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0b57c0 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0b5d80 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f13d8040070 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.046+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f13d804bab0 con 0x7f13ec0cec20 2024-03-04T12:11:32.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.047+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec074f60 con 0x7f13ec0cec20 2024-03-04T12:11:32.050 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.049+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f13d8033030 con 0x7f13ec0cec20 2024-03-04T12:11:32.073 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.072+0000 7fc40981d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T12:11:32.083 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.083+0000 7fa5ebe39040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T12:11:32.131 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.130+0000 7fa5ebe39040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T12:11:32.159 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.159+0000 7fc40981d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T12:11:32.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.199+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3c655fef-c6c1-4080-8af7-217ce635c225", "id": 4} v 0) v1 -- 0x7f13ec0b77c0 con 0x7f13ec0cec20 2024-03-04T12:11:32.205 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.204+0000 7f13e97fa640 1 -- 172.21.15.191:0/116576818 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c655fef-c6c1-4080-8af7-217ce635c225", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13d802f8e0 con 0x7f13ec0cec20 2024-03-04T12:11:32.205 INFO:teuthology.orchestra.run.smithi191.stdout:4 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 msgr2=0x7f13ec0ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8004730 comp rx=0 tx=0).stop 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 shutdown_connections 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13ec161390 0x7f13ec0ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13ec0cec20 0x7f13ec0ac8f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 --2- 172.21.15.191:0/116576818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec06c060 0x7f13ec0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 >> 172.21.15.191:0/116576818 conn(0x7f13ec066dc0 msgr2=0x7f13ec0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 shutdown_connections 2024-03-04T12:11:32.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.206+0000 7f13f3116640 1 -- 172.21.15.191:0/116576818 wait complete. 2024-03-04T12:11:32.219 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 4b5c175f-dfcd-43c8-b96d-c65760b94902 5 2024-03-04T12:11:32.227 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.227+0000 7fc40981d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T12:11:32.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.319+0000 7f284e17e640 1 Processor -- start 2024-03-04T12:11:32.320 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.319+0000 7f284e17e640 1 -- start start 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f284e17e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 0x7f284806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f284e17e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284806bb60 con 0x7f284815df00 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f284e17e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480701d0 con 0x7f284806b700 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f284e17e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480704b0 con 0x7f28480d0150 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2846ffd640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f284806b700 0x7f28480cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35200/0 (socket says 172.21.15.191:35200) 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2847fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 0x7f284806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.321 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2846ffd640 1 -- 172.21.15.191:0/313960680 learned_addr learned my addr 172.21.15.191:0/313960680 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-04T12:11:32.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f28467fc640 1 -- 172.21.15.191:0/313960680 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587137590 0 0) 0x7f28480701d0 con 0x7f284806b700 2024-03-04T12:11:32.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f28467fc640 1 -- 172.21.15.191:0/313960680 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2830003520 con 0x7f284806b700 2024-03-04T12:11:32.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2847fff640 1 -- 172.21.15.191:0/313960680 >> v1:172.21.15.191:6789/0 conn(0x7f284806b700 legacy=0x7f28480cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2847fff640 1 -- 172.21.15.191:0/313960680 >> v1:172.21.15.148:6789/0 conn(0x7f284815df00 legacy=0x7f28480cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.322 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.320+0000 7f2847fff640 1 -- 172.21.15.191:0/313960680 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480cc8e0 con 0x7f28480d0150 2024-03-04T12:11:32.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.321+0000 7f2847fff640 1 --2- 172.21.15.191:0/313960680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 0x7f284806fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2840009dc0 tx=0x7f284002be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ba4560075a7d56 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.321+0000 7f28467fc640 1 -- 172.21.15.191:0/313960680 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840039070 con 0x7f28480d0150 2024-03-04T12:11:32.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.321+0000 7f28467fc640 1 -- 172.21.15.191:0/313960680 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284002cc30 con 0x7f28480d0150 2024-03-04T12:11:32.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.321+0000 7f28467fc640 1 -- 172.21.15.191:0/313960680 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840034440 con 0x7f28480d0150 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 -- 172.21.15.191:0/313960680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 msgr2=0x7f284806fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 --2- 172.21.15.191:0/313960680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 0x7f284806fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2840009dc0 tx=0x7f284002be80 comp rx=0 tx=0).stop 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 -- 172.21.15.191:0/313960680 shutdown_connections 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 --2- 172.21.15.191:0/313960680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28480d0150 0x7f284806fa80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 -- 172.21.15.191:0/313960680 >> 172.21.15.191:0/313960680 conn(0x7f28480b2250 msgr2=0x7f28480b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 -- 172.21.15.191:0/313960680 shutdown_connections 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.322+0000 7f284e17e640 1 -- 172.21.15.191:0/313960680 wait complete. 2024-03-04T12:11:32.324 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.323+0000 7f284e17e640 1 Processor -- start 2024-03-04T12:11:32.326 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.323+0000 7f284e17e640 1 -- start start 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 0x7f28480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 0x7f28480c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480cd3c0 con 0x7f284815df00 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480ccea0 con 0x7f28480d0150 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480cd540 con 0x7f284806b700 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f28477fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 0x7f28480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f28477fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 0x7f28480b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46942/0 (socket says 172.21.15.191:46942) 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55398/0 (socket says 172.21.15.191:55398) 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f28477fe640 1 -- 172.21.15.191:0/2554425818 learned_addr learned my addr 172.21.15.191:0/2554425818 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2847fff640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 0x7f28480c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.327 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 -- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 msgr2=0x7f28480b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 0x7f28480b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 -- 172.21.15.191:0/2554425818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 msgr2=0x7f28480c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 0x7f28480c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 -- 172.21.15.191:0/2554425818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480b3cf0 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2847fff640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 0x7f28480c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f2846ffd640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c430 tx=0x7f283c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c017070 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c005ce0 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28480b3fd0 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.324+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0072d0 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.325+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28480b4870 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.325+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f283c01a070 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.325+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f283c01cc10 con 0x7f28480d0150 2024-03-04T12:11:32.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.325+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2848074f60 con 0x7f28480d0150 2024-03-04T12:11:32.329 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.327+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f283c007470 con 0x7f28480d0150 2024-03-04T12:11:32.400 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.399+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4b5c175f-dfcd-43c8-b96d-c65760b94902", "id": 5} v 0) v1 -- 0x7f284815bd70 con 0x7f28480d0150 2024-03-04T12:11:32.406 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.405+0000 7f282ffff640 1 -- 172.21.15.191:0/2554425818 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b5c175f-dfcd-43c8-b96d-c65760b94902", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f283c0076b0 con 0x7f28480d0150 2024-03-04T12:11:32.406 INFO:teuthology.orchestra.run.smithi191.stdout:5 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 msgr2=0x7f2848152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c430 tx=0x7f283c00c8f0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 shutdown_connections 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284815df00 0x7f28480c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28480d0150 0x7f2848152920 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 --2- 172.21.15.191:0/2554425818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284806b700 0x7f28480b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 >> 172.21.15.191:0/2554425818 conn(0x7f28480b2250 msgr2=0x7f28480b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 shutdown_connections 2024-03-04T12:11:32.408 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.406+0000 7f284e17e640 1 -- 172.21.15.191:0/2554425818 wait complete. 2024-03-04T12:11:32.417 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 35c6c230-5f91-4544-a15c-b43b4021a946 6 2024-03-04T12:11:32.510 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.509+0000 7fa5ebe39040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T12:11:32.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.558+0000 7f400ab7e640 1 Processor -- start 2024-03-04T12:11:32.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.558+0000 7f400ab7e640 1 -- start start 2024-03-04T12:11:32.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400ab7e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400ab7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400ab7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400415c790 con 0x7f40040ccbd0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400ab7e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400415d2f0 con 0x7f40040ceec0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400ab7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400415d6d0 con 0x7f400415c2e0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f4003fff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46946/0 (socket says 172.21.15.191:46946) 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f4003fff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55406/0 (socket says 172.21.15.191:55406) 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 -- 172.21.15.191:0/1092373395 learned_addr learned my addr 172.21.15.191:0/1092373395 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 -- 172.21.15.191:0/1092373395 >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 msgr2=0x7f40040b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 --2- 172.21.15.191:0/1092373395 >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 -- 172.21.15.191:0/1092373395 >> v1:172.21.15.148:6789/0 conn(0x7f40040ccbd0 legacy=0x7f40040ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f400911b640 1 -- 172.21.15.191:0/1092373395 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400415ca50 con 0x7f400415c2e0 2024-03-04T12:11:32.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.559+0000 7f4003fff640 1 --2- 172.21.15.191:0/1092373395 >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.560+0000 7f400911b640 1 --2- 172.21.15.191:0/1092373395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009e50 tx=0x7f3ffc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca08e723f4dd9c22 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.560+0000 7f40037fe640 1 -- 172.21.15.191:0/1092373395 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc02cb00 con 0x7f400415c2e0 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.560+0000 7f40037fe640 1 -- 172.21.15.191:0/1092373395 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc02cca0 con 0x7f400415c2e0 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.560+0000 7f40037fe640 1 -- 172.21.15.191:0/1092373395 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc034610 con 0x7f400415c2e0 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 -- 172.21.15.191:0/1092373395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 msgr2=0x7f40040b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 --2- 172.21.15.191:0/1092373395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009e50 tx=0x7f3ffc02bf60 comp rx=0 tx=0).stop 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 -- 172.21.15.191:0/1092373395 shutdown_connections 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 --2- 172.21.15.191:0/1092373395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040b0560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.563 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 --2- 172.21.15.191:0/1092373395 >> v2:172.21.15.191:3300/0 conn(0x7f40040ceec0 0x7f40040b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 -- 172.21.15.191:0/1092373395 >> 172.21.15.191:0/1092373395 conn(0x7f4004066dc0 msgr2=0x7f40040671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 -- 172.21.15.191:0/1092373395 shutdown_connections 2024-03-04T12:11:32.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.561+0000 7f400ab7e640 1 -- 172.21.15.191:0/1092373395 wait complete. 2024-03-04T12:11:32.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.562+0000 7f400ab7e640 1 Processor -- start 2024-03-04T12:11:32.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.562+0000 7f400ab7e640 1 -- start start 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40040ceec0 0x7f40040d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040ccfb0 con 0x7f40040ceec0 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400415d010 con 0x7f40040ccbd0 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400ab7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40040b0aa0 con 0x7f400415c2e0 2024-03-04T12:11:32.566 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400911b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55420/0 (socket says 172.21.15.191:55420) 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 -- 172.21.15.191:0/661920371 learned_addr learned my addr 172.21.15.191:0/661920371 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f4003fff640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40040ceec0 0x7f40040d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 -- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 msgr2=0x7f40040c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 -- 172.21.15.191:0/661920371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40040ceec0 msgr2=0x7f40040d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40040ceec0 0x7f40040d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 -- 172.21.15.191:0/661920371 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004156e10 con 0x7f40040ccbd0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400911b640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.563+0000 7f400891a640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c460 tx=0x7f3ff800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8016020 con 0x7f40040ccbd0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff80052f0 con 0x7f40040ccbd0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40041570f0 con 0x7f40040ccbd0 2024-03-04T12:11:32.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8007260 con 0x7f40040ccbd0 2024-03-04T12:11:32.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40041576b0 con 0x7f40040ccbd0 2024-03-04T12:11:32.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3ff80067d0 con 0x7f40040ccbd0 2024-03-04T12:11:32.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.564+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f3ff8012cc0 con 0x7f40040ccbd0 2024-03-04T12:11:32.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.565+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4004074f60 con 0x7f40040ccbd0 2024-03-04T12:11:32.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.567+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3ff8020020 con 0x7f40040ccbd0 2024-03-04T12:11:32.627 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.626+0000 7fc40981d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T12:11:32.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.639+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35c6c230-5f91-4544-a15c-b43b4021a946", "id": 6} v 0) v1 -- 0x7f40040b0ed0 con 0x7f40040ccbd0 2024-03-04T12:11:32.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.645+0000 7f40017fa640 1 -- 172.21.15.191:0/661920371 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35c6c230-5f91-4544-a15c-b43b4021a946", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ff8006cf0 con 0x7f40040ccbd0 2024-03-04T12:11:32.646 INFO:teuthology.orchestra.run.smithi191.stdout:6 2024-03-04T12:11:32.646 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.646+0000 7fa5ebe39040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T12:11:32.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 msgr2=0x7f40040cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c460 tx=0x7f3ff800c920 comp rx=0 tx=0).stop 2024-03-04T12:11:32.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 shutdown_connections 2024-03-04T12:11:32.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f400415c2e0 0x7f40040c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40040ceec0 0x7f40040d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 --2- 172.21.15.191:0/661920371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40040ccbd0 0x7f40040cb290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 >> 172.21.15.191:0/661920371 conn(0x7f4004066dc0 msgr2=0x7f40040b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 shutdown_connections 2024-03-04T12:11:32.648 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.646+0000 7f400ab7e640 1 -- 172.21.15.191:0/661920371 wait complete. 2024-03-04T12:11:32.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new b962274a-4949-4442-a2fc-527f35e5b1d6 7 2024-03-04T12:11:32.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.757+0000 7f316f9ab640 1 Processor -- start 2024-03-04T12:11:32.758 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316f9ab640 1 -- start start 2024-03-04T12:11:32.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316f9ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f316806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.759 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316f9ab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316806bb60 con 0x7f316806b700 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316f9ab640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680701d0 con 0x7f316815df00 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316f9ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680704b0 con 0x7f31680d0150 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316d747640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f316815df00 0x7f31680cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35204/0 (socket says 172.21.15.191:35204) 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316df48640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f316806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.758+0000 7f316d747640 1 -- 172.21.15.191:0/1976652484 learned_addr learned my addr 172.21.15.191:0/1976652484 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.759+0000 7f315ffff640 1 -- 172.21.15.191:0/1976652484 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750063311 0 0) 0x7f31680701d0 con 0x7f316815df00 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.759+0000 7f316df48640 1 -- 172.21.15.191:0/1976652484 >> v1:172.21.15.191:6789/0 conn(0x7f316815df00 legacy=0x7f31680cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.759+0000 7f316df48640 1 -- 172.21.15.191:0/1976652484 >> v1:172.21.15.148:6789/0 conn(0x7f316806b700 legacy=0x7f31680cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:32.760 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.759+0000 7f316df48640 1 -- 172.21.15.191:0/1976652484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680cc8e0 con 0x7f31680d0150 2024-03-04T12:11:32.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.759+0000 7f316df48640 1 --2- 172.21.15.191:0/1976652484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f316806fa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3160009dc0 tx=0x7f316002be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b193a583e278b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f315ffff640 1 -- 172.21.15.191:0/1976652484 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160039070 con 0x7f31680d0150 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f315ffff640 1 -- 172.21.15.191:0/1976652484 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316002cbb0 con 0x7f31680d0150 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f315ffff640 1 -- 172.21.15.191:0/1976652484 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160034440 con 0x7f31680d0150 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 -- 172.21.15.191:0/1976652484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 msgr2=0x7f316806fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 --2- 172.21.15.191:0/1976652484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f316806fa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3160009dc0 tx=0x7f316002be00 comp rx=0 tx=0).stop 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 -- 172.21.15.191:0/1976652484 shutdown_connections 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 --2- 172.21.15.191:0/1976652484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f316806fa80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 -- 172.21.15.191:0/1976652484 >> 172.21.15.191:0/1976652484 conn(0x7f31680b2250 msgr2=0x7f31680b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 -- 172.21.15.191:0/1976652484 shutdown_connections 2024-03-04T12:11:32.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.760+0000 7f316f9ab640 1 -- 172.21.15.191:0/1976652484 wait complete. 2024-03-04T12:11:32.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.761+0000 7f316f9ab640 1 Processor -- start 2024-03-04T12:11:32.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.761+0000 7f316f9ab640 1 -- start start 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f316806b700 0x7f31681608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316815df00 0x7f31680aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680cd3c0 con 0x7f316815df00 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680ccea0 con 0x7f316806b700 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316f9ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680cd540 con 0x7f31680d0150 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316cf46640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316d747640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f316806b700 0x7f31681608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316cf46640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:46968/0 (socket says 172.21.15.191:46968) 2024-03-04T12:11:32.765 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316cf46640 1 -- 172.21.15.191:0/3210941960 learned_addr learned my addr 172.21.15.191:0/3210941960 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.762+0000 7f316df48640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316815df00 0x7f31680aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 -- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f316806b700 msgr2=0x7f31681608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f316806b700 0x7f31681608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 -- 172.21.15.191:0/3210941960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316815df00 msgr2=0x7f31680aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316815df00 0x7f31680aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 -- 172.21.15.191:0/3210941960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31681f3950 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316cf46640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f316400c460 tx=0x7f316400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164016020 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164012870 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31681f3c10 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164012c60 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681f41d0 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3164019020 con 0x7f31680d0150 2024-03-04T12:11:32.766 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.763+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f316401a3b0 con 0x7f31680d0150 2024-03-04T12:11:32.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.764+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3168074f60 con 0x7f31680d0150 2024-03-04T12:11:32.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.766+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3164020030 con 0x7f31680d0150 2024-03-04T12:11:32.773 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.772+0000 7fc40981d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T12:11:32.775 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.775+0000 7fa5ebe39040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T12:11:32.822 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.821+0000 7fa5ebe39040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T12:11:32.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.849+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b962274a-4949-4442-a2fc-527f35e5b1d6", "id": 7} v 0) v1 -- 0x7f31680afbb0 con 0x7f31680d0150 2024-03-04T12:11:32.858 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.857+0000 7f315dffb640 1 -- 172.21.15.191:0/3210941960 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b962274a-4949-4442-a2fc-527f35e5b1d6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3164012e00 con 0x7f31680d0150 2024-03-04T12:11:32.858 INFO:teuthology.orchestra.run.smithi191.stdout:7 2024-03-04T12:11:32.860 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 msgr2=0x7f31680ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f316400c460 tx=0x7f316400c920 comp rx=0 tx=0).stop 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 shutdown_connections 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f316815df00 0x7f31680aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31680d0150 0x7f31680ae540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 --2- 172.21.15.191:0/3210941960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f316806b700 0x7f31681608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 >> 172.21.15.191:0/3210941960 conn(0x7f31680b2250 msgr2=0x7f31680b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 shutdown_connections 2024-03-04T12:11:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T12:11:32.859+0000 7f316f9ab640 1 -- 172.21.15.191:0/3210941960 wait complete. 2024-03-04T12:11:32.874 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-04T12:11:32.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-04T12:11:32.876 INFO:tasks.ceph.osd.0:Started 2024-03-04T12:11:32.876 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-04T12:11:32.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-04T12:11:32.878 INFO:tasks.ceph.osd.1:Started 2024-03-04T12:11:32.878 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-04T12:11:32.879 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-04T12:11:32.880 INFO:tasks.ceph.osd.2:Started 2024-03-04T12:11:32.880 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-04T12:11:32.881 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-04T12:11:32.883 INFO:tasks.ceph.osd.3:Started 2024-03-04T12:11:32.883 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-04T12:11:32.883 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-04T12:11:32.885 INFO:tasks.ceph.osd.4:Started 2024-03-04T12:11:32.886 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-04T12:11:32.886 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-04T12:11:32.888 INFO:tasks.ceph.osd.5:Started 2024-03-04T12:11:32.888 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-04T12:11:32.888 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-04T12:11:32.890 INFO:tasks.ceph.osd.6:Started 2024-03-04T12:11:32.890 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-04T12:11:32.890 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-04T12:11:32.894 INFO:tasks.ceph.osd.7:Started 2024-03-04T12:11:32.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:32.905 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.905+0000 7fc40981d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T12:11:32.969 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:32.968+0000 7fa5ebe39040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T12:11:32.978 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:32.978+0000 7fc40981d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T12:11:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.012+0000 7f95c1553640 1 Processor -- start 2024-03-04T12:11:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.012+0000 7f95c1553640 1 -- start start 2024-03-04T12:11:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95c1553640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95c1553640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc06ae50 con 0x7f95bc0cfae0 2024-03-04T12:11:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95c1553640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc06b130 con 0x7f95bc0cc8e0 2024-03-04T12:11:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95c1553640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc06b410 con 0x7f95bc0d1290 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95baffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f95bc0cfae0 0x7f95bc0cadc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39546/0 (socket says 172.21.15.148:39546) 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95baffd640 1 -- 172.21.15.148:0/1847041593 learned_addr learned my addr 172.21.15.148:0/1847041593 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95bb7fe640 1 --2- 172.21.15.148:0/1847041593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.013+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360587508 0 0) 0x7f95bc06ae50 con 0x7f95bc0cfae0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a4003520 con 0x7f95bc0cfae0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829443309 0 0) 0x7f95bc06b130 con 0x7f95bc0cc8e0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95bc06ae50 con 0x7f95bc0cc8e0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95bb7fe640 1 -- 172.21.15.148:0/1847041593 >> v1:172.21.15.191:6789/0 conn(0x7f95bc0cc8e0 legacy=0x7f95bc0cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95bb7fe640 1 -- 172.21.15.148:0/1847041593 >> v1:172.21.15.148:6789/0 conn(0x7f95bc0cfae0 legacy=0x7f95bc0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95bb7fe640 1 -- 172.21.15.148:0/1847041593 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc06b6f0 con 0x7f95bc0d1290 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.014+0000 7f95bb7fe640 1 --2- 172.21.15.148:0/1847041593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0cd440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009dc0 tx=0x7f95b402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f495574b1ee04fa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4039070 con 0x7f95bc0d1290 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b402ccd0 con 0x7f95bc0d1290 2024-03-04T12:11:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95b9ffb640 1 -- 172.21.15.148:0/1847041593 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4034640 con 0x7f95bc0d1290 2024-03-04T12:11:33.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 -- 172.21.15.148:0/1847041593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 msgr2=0x7f95bc0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 --2- 172.21.15.148:0/1847041593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0cd440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009dc0 tx=0x7f95b402bf20 comp rx=0 tx=0).stop 2024-03-04T12:11:33.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 -- 172.21.15.148:0/1847041593 shutdown_connections 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 --2- 172.21.15.148:0/1847041593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0cd440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 -- 172.21.15.148:0/1847041593 >> 172.21.15.148:0/1847041593 conn(0x7f95bc0d0770 msgr2=0x7f95bc0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 -- 172.21.15.148:0/1847041593 shutdown_connections 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.015+0000 7f95c1553640 1 -- 172.21.15.148:0/1847041593 wait complete. 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 Processor -- start 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 -- start start 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95bc0cc8e0 0x7f95bc160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0cd980 con 0x7f95bc0cfae0 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc06b9d0 con 0x7f95bc0cc8e0 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95c1553640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0cdb00 con 0x7f95bc0d1290 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95ba7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95ba7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53046/0 (socket says 172.21.15.148:53046) 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95ba7fc640 1 -- 172.21.15.148:0/3350739922 learned_addr learned my addr 172.21.15.148:0/3350739922 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95baffd640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95bc0cc8e0 0x7f95bc160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.017+0000 7f95bb7fe640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 -- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 msgr2=0x7f95bc0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 -- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95bc0cc8e0 msgr2=0x7f95bc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95bc0cc8e0 0x7f95bc160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 -- 172.21.15.148:0/3350739922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0c7bb0 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95ba7fc640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c530 tx=0x7f95ac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0052d0 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac005490 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac012870 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc160040 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc0c8700 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f95ac012a10 con 0x7f95bc0cfae0 2024-03-04T12:11:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.018+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f95ac01bc00 con 0x7f95bc0cfae0 2024-03-04T12:11:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.019+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc06cd70 con 0x7f95bc0cfae0 2024-03-04T12:11:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.021+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f95ac01b4f0 con 0x7f95bc0cfae0 2024-03-04T12:11:33.071 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:33.070+0000 7fa5ebe39040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T12:11:33.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.082+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95bc0cb120 con 0x7f95bc0cfae0 2024-03-04T12:11:33.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.083+0000 7f95a37fe640 1 -- 172.21.15.148:0/3350739922 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f95ac01b6d0 con 0x7f95bc0cfae0 2024-03-04T12:11:33.084 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:33.084 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":10,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:32.851843+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 msgr2=0x7f95bc14cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c530 tx=0x7f95ac00c9f0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 shutdown_connections 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95bc0d1290 0x7f95bc0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc0cfae0 0x7f95bc14cbd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 --2- 172.21.15.148:0/3350739922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95bc0cc8e0 0x7f95bc160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 >> 172.21.15.148:0/3350739922 conn(0x7f95bc0d0770 msgr2=0x7f95bc0738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:33.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 shutdown_connections 2024-03-04T12:11:33.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:33.084+0000 7f95c1553640 1 -- 172.21.15.148:0/3350739922 wait complete. 2024-03-04T12:11:33.094 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:33.093+0000 7fc40981d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T12:11:33.095 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-04T12:11:33.095 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-04T12:11:33.189 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:33.188+0000 7fc40981d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T12:11:33.228 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:33.228+0000 7fa5ebe39040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T12:11:33.272 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:33.271+0000 7fa5ebe39040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T12:11:33.345 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:33.344+0000 7fc40981d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T12:11:33.388 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:33.387+0000 7fc40981d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T12:11:33.426 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:33.425+0000 7fa5ebe39040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T12:11:33.482 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T12:11:33.481+0000 7f382f7dd640 -1 Falling back to public interface 2024-03-04T12:11:33.485 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:11:33.484+0000 7fe463011640 -1 Falling back to public interface 2024-03-04T12:11:33.491 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T12:11:33.490+0000 7f6884bf6640 -1 Falling back to public interface 2024-03-04T12:11:33.493 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T12:11:33.492+0000 7f21a6e1f640 -1 Falling back to public interface 2024-03-04T12:11:33.498 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:11:33.497+0000 7fea38e14640 -1 Falling back to public interface 2024-03-04T12:11:33.498 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:11:33.497+0000 7f19b8cf9640 -1 Falling back to public interface 2024-03-04T12:11:33.509 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T12:11:33.509+0000 7f1dffd24640 -1 Falling back to public interface 2024-03-04T12:11:33.512 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:11:33.511+0000 7fb0f3e14640 -1 Falling back to public interface 2024-03-04T12:11:33.548 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:33.547+0000 7fc40981d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T12:11:33.900 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:33.991 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.990+0000 7f77773b9640 1 Processor -- start 2024-03-04T12:11:33.991 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 -- start start 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 0x7f77700b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7770162420 0x7f777015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777015eee0 con 0x7f77700aff40 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7770161940 con 0x7f7770162420 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.991+0000 7f77773b9640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7770161c20 con 0x7f77700afb60 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775956640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7770162420 0x7f777015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7774954640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f77700aff40 0x7f777015e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39560/0 (socket says 172.21.15.148:39560) 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7774954640 1 -- 172.21.15.148:0/507513688 learned_addr learned my addr 172.21.15.148:0/507513688 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:33.993 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775155640 1 --2- 172.21.15.148:0/507513688 >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 0x7f77700b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.994 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775956640 1 -- 172.21.15.148:0/507513688 >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 msgr2=0x7f77700b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:33.994 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775956640 1 --2- 172.21.15.148:0/507513688 >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 0x7f77700b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.994 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775956640 1 -- 172.21.15.148:0/507513688 >> v1:172.21.15.148:6789/0 conn(0x7f77700aff40 legacy=0x7f777015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.994 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775956640 1 -- 172.21.15.148:0/507513688 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770161f00 con 0x7f7770162420 2024-03-04T12:11:33.994 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.992+0000 7f7775155640 1 --2- 172.21.15.148:0/507513688 >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 0x7f77700b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:33.995 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f7775956640 1 --2- 172.21.15.148:0/507513688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7770162420 0x7f777015e8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7768009d90 tx=0x7f776802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5eff2f8e48ada server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:33.995 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f7767fff640 1 -- 172.21.15.148:0/507513688 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768039070 con 0x7f7770162420 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f7767fff640 1 -- 172.21.15.148:0/507513688 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776802cbd0 con 0x7f7770162420 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f7767fff640 1 -- 172.21.15.148:0/507513688 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768034440 con 0x7f7770162420 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 -- 172.21.15.148:0/507513688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7770162420 msgr2=0x7f777015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 --2- 172.21.15.148:0/507513688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7770162420 0x7f777015e8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7768009d90 tx=0x7f776802be20 comp rx=0 tx=0).stop 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 -- 172.21.15.148:0/507513688 shutdown_connections 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 --2- 172.21.15.148:0/507513688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7770162420 0x7f777015e8c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 --2- 172.21.15.148:0/507513688 >> v2:172.21.15.191:3300/0 conn(0x7f77700afb60 0x7f77700b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.993+0000 7f77773b9640 1 -- 172.21.15.148:0/507513688 >> 172.21.15.148:0/507513688 conn(0x7f77700b2400 msgr2=0x7f77700b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.994+0000 7f77773b9640 1 -- 172.21.15.148:0/507513688 shutdown_connections 2024-03-04T12:11:33.996 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.994+0000 7f77773b9640 1 -- 172.21.15.148:0/507513688 wait complete. 2024-03-04T12:11:33.997 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 Processor -- start 2024-03-04T12:11:33.997 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 -- start start 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77700afb60 0x7f7770153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7770162420 0x7f77701f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700b4c30 con 0x7f77700aff40 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700b4ab0 con 0x7f77700afb60 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.995+0000 7f77773b9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77701579c0 con 0x7f7770162420 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:33.999 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53070/0 (socket says 172.21.15.148:53070) 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 -- 172.21.15.148:0/2065026641 learned_addr learned my addr 172.21.15.148:0/2065026641 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7775155640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77700afb60 0x7f7770153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7775956640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7770162420 0x7f77701f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 -- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7770162420 msgr2=0x7f77701f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7770162420 0x7f77701f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 -- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77700afb60 msgr2=0x7f7770153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77700afb60 0x7f7770153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 -- 172.21.15.148:0/2065026641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700cc980 con 0x7f77700aff40 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7774954640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f776000c3a0 tx=0x7f776000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760017070 con 0x7f77700aff40 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7760005ce0 con 0x7f77700aff40 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77700cc6a0 con 0x7f77700aff40 2024-03-04T12:11:34.000 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600072f0 con 0x7f77700aff40 2024-03-04T12:11:34.001 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77700cdb40 con 0x7f77700aff40 2024-03-04T12:11:34.001 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.996+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f776001a070 con 0x7f77700aff40 2024-03-04T12:11:34.001 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.997+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7760012cd0 con 0x7f77700aff40 2024-03-04T12:11:34.001 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:33.997+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777004f090 con 0x7f77700aff40 2024-03-04T12:11:34.001 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.000+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7760007490 con 0x7f77700aff40 2024-03-04T12:11:34.062 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.062+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77700b3230 con 0x7f77700aff40 2024-03-04T12:11:34.063 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.062+0000 7f7765ffb640 1 -- 172.21.15.148:0/2065026641 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f7760007670 con 0x7f77700aff40 2024-03-04T12:11:34.063 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-04T12:11:34.063 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":10,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:32.851843+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 msgr2=0x7f77700ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f776000c3a0 tx=0x7f776000c860 comp rx=0 tx=0).stop 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 shutdown_connections 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7770162420 0x7f77701f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700aff40 0x7f77700ce340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:34.065 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 --2- 172.21.15.148:0/2065026641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77700afb60 0x7f7770153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:34.066 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 >> 172.21.15.148:0/2065026641 conn(0x7f77700b2400 msgr2=0x7f77700b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:34.066 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 shutdown_connections 2024-03-04T12:11:34.066 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:34.063+0000 7f77773b9640 1 -- 172.21.15.148:0/2065026641 wait complete. 2024-03-04T12:11:34.073 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T12:11:34.247 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:34.246+0000 7fa5ebe39040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T12:11:34.373 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:34.372+0000 7fc40981d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T12:11:34.524 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:34.523+0000 7fa5ebe39040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T12:11:34.575 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:34.574+0000 7fa5ebe39040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T12:11:34.652 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:34.651+0000 7fc40981d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T12:11:34.692 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:34.692+0000 7fa5ebe39040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T12:11:34.704 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:34.703+0000 7fc40981d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T12:11:34.825 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-04T12:11:34.824+0000 7fc40981d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T12:11:35.717 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:35.716+0000 7fa58cb9a640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-04T12:11:35.717 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T12:11:35.716+0000 7fa58cb9a640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-04T12:11:36.973 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T12:11:36.972+0000 7f382f7dd640 -1 osd.4 0 log_to_monitors true 2024-03-04T12:11:36.978 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:11:36.976+0000 7fe463011640 -1 osd.0 0 log_to_monitors true 2024-03-04T12:11:36.986 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T12:11:36.985+0000 7f6884bf6640 -1 osd.6 0 log_to_monitors true 2024-03-04T12:11:36.992 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T12:11:36.991+0000 7f21a6e1f640 -1 osd.5 0 log_to_monitors true 2024-03-04T12:11:37.017 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T12:11:37.016+0000 7f1dffd24640 -1 osd.7 0 log_to_monitors true 2024-03-04T12:11:37.019 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:11:37.017+0000 7fb0f3e14640 -1 osd.1 0 log_to_monitors true 2024-03-04T12:11:37.052 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:11:37.050+0000 7fea38e14640 -1 osd.2 0 log_to_monitors true 2024-03-04T12:11:37.063 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:11:37.062+0000 7f19b8cf9640 -1 osd.3 0 log_to_monitors true 2024-03-04T12:11:40.752 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:11:40.751+0000 7fea347ec640 -1 osd.2 0 waiting for initial osdmap 2024-03-04T12:11:40.769 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:11:40.768+0000 7fea2c60c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.772 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:11:40.771+0000 7fe45e90f640 -1 osd.0 0 waiting for initial osdmap 2024-03-04T12:11:40.785 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:11:40.785+0000 7fe45672f640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.813 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:11:40.812+0000 7f19b4c8f640 -1 osd.3 0 waiting for initial osdmap 2024-03-04T12:11:40.823 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T12:11:40.822+0000 7f21a2888640 -1 osd.5 0 waiting for initial osdmap 2024-03-04T12:11:40.825 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T12:11:40.824+0000 7f6880b8c640 -1 osd.6 0 waiting for initial osdmap 2024-03-04T12:11:40.830 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:11:40.830+0000 7f19acaaf640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.837 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T12:11:40.836+0000 7f219a6a8640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.844 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T12:11:40.843+0000 7f68789ac640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.870 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T12:11:40.869+0000 7f382b773640 -1 osd.4 0 waiting for initial osdmap 2024-03-04T12:11:40.880 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:40.880 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T12:11:40.879+0000 7f3823593640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.881 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T12:11:40.879+0000 7f1dfc4ce640 -1 osd.7 0 waiting for initial osdmap 2024-03-04T12:11:40.907 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T12:11:40.906+0000 7f1df3ada640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:40.974 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.973+0000 7f3d87081640 1 Processor -- start 2024-03-04T12:11:40.974 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.973+0000 7f3d87081640 1 -- start start 2024-03-04T12:11:40.979 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 0x7f3d800b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d800b9a30 con 0x7f3d800d34f0 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d800d2450 con 0x7f3d8015df70 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d87081640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800d2730 con 0x7f3d8015db90 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d8561e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d8561e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53178/0 (socket says 172.21.15.148:53178) 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d8561e640 1 -- 172.21.15.148:0/930583486 learned_addr learned my addr 172.21.15.148:0/930583486 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/930583486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d7ffff640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 0x7f3d800b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 -- 172.21.15.148:0/930583486 >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 msgr2=0x7f3d800b94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:40.980 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 0x7f3d800b94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 -- 172.21.15.148:0/930583486 >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 msgr2=0x7f3d800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d84e1d640 1 -- 172.21.15.148:0/930583486 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800d2a10 con 0x7f3d8015db90 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d8561e640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.975+0000 7f3d7ffff640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 0x7f3d800b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.976+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/930583486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d800d1de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d800d4a70 tx=0x7f3d7402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4b988d08ec0572 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.976+0000 7f3d7f7fe640 1 -- 172.21.15.148:0/930583486 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74038070 con 0x7f3d8015db90 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.976+0000 7f3d7f7fe640 1 -- 172.21.15.148:0/930583486 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74031700 con 0x7f3d8015db90 2024-03-04T12:11:40.981 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.976+0000 7f3d7f7fe640 1 -- 172.21.15.148:0/930583486 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031ad0 con 0x7f3d8015db90 2024-03-04T12:11:40.984 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 -- 172.21.15.148:0/930583486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 msgr2=0x7f3d800d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 --2- 172.21.15.148:0/930583486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d800d1de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d800d4a70 tx=0x7f3d7402b7a0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 -- 172.21.15.148:0/930583486 shutdown_connections 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.148:3300/0 conn(0x7f3d800d34f0 0x7f3d800ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 --2- 172.21.15.148:0/930583486 >> v2:172.21.15.191:3300/0 conn(0x7f3d8015df70 0x7f3d800b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 --2- 172.21.15.148:0/930583486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d800d1de0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 -- 172.21.15.148:0/930583486 >> 172.21.15.148:0/930583486 conn(0x7f3d800d0490 msgr2=0x7f3d800d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 -- 172.21.15.148:0/930583486 shutdown_connections 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.977+0000 7f3d87081640 1 -- 172.21.15.148:0/930583486 wait complete. 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.979+0000 7f3d87081640 1 Processor -- start 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.979+0000 7f3d87081640 1 -- start start 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 0x7f3d80146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d801470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.985 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800ced20 con 0x7f3d8015df70 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800d2fd0 con 0x7f3d800d34f0 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d87081640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800ceea0 con 0x7f3d8015db90 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d8561e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d8561e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53186/0 (socket says 172.21.15.148:53186) 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d8561e640 1 -- 172.21.15.148:0/3224642610 learned_addr learned my addr 172.21.15.148:0/3224642610 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 0x7f3d80146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.980+0000 7f3d7ffff640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d801470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 -- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 msgr2=0x7f3d801470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d801470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 -- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 msgr2=0x7f3d80146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 0x7f3d80146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 -- 172.21.15.148:0/3224642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8015f620 con 0x7f3d8015df70 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 0x7f3d80146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:40.986 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d8561e640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800eed0 tx=0x7f3d7800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800bb10 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800bcb0 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800be60 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8015f900 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.981+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d80146d60 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.982+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8004f090 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.982+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 4) v1 ==== 89722+0+0 (secure 0 0 0) 0x7f3d7801b030 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.983+0000 7f3d7d7fa640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 0x7f3d6406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.983+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 0x7f3d6406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.983+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f3d78089240 con 0x7f3d8015df70 2024-03-04T12:11:40.987 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.983+0000 7f3d84e1d640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 0x7f3d6406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d74035000 tx=0x7f3d74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:40.992 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:40.991+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d78022020 con 0x7f3d8015df70 2024-03-04T12:11:41.003 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:11:41.002+0000 7fb0ef761640 -1 osd.1 0 waiting for initial osdmap 2024-03-04T12:11:41.017 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:11:41.016+0000 7fb0e7581640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T12:11:41.167 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.166+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d8015c450 con 0x7f3d8015df70 2024-03-04T12:11:41.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.167+0000 7f3d7d7fa640 1 -- 172.21.15.148:0/3224642610 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f3d78058fc0 con 0x7f3d8015df70 2024-03-04T12:11:41.168 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-04T12:11:41.168 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":12,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:38.722256+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 msgr2=0x7f3d6406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 0x7f3d6406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d74035000 tx=0x7f3d74008030 comp rx=0 tx=0).stop 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 msgr2=0x7f3d801598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800eed0 tx=0x7f3d7800b480 comp rx=0 tx=0).stop 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 shutdown_connections 2024-03-04T12:11:41.170 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d6406b440 0x7f3d6406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d8015df70 0x7f3d801598e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8015db90 0x7f3d801470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 --2- 172.21.15.148:0/3224642610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d800d34f0 0x7f3d80146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 >> 172.21.15.148:0/3224642610 conn(0x7f3d800d0490 msgr2=0x7f3d800b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 shutdown_connections 2024-03-04T12:11:41.171 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:41.169+0000 7f3d87081640 1 -- 172.21.15.148:0/3224642610 wait complete. 2024-03-04T12:11:41.179 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T12:11:47.986 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:48.086 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.086+0000 7f43c46fc640 1 Processor -- start 2024-03-04T12:11:48.087 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.086+0000 7f43c46fc640 1 -- start start 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c46fc640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43bc0cc220 0x7f43bc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c46fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43bc0cad50 0x7f43bc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c46fc640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0cdfe0 con 0x7f43bc0cc8e0 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c46fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc0ce620 con 0x7f43bc0cad50 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c46fc640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc06ff60 con 0x7f43bc0cc220 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43bc0cad50 0x7f43bc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43bc0cad50 0x7f43bc0cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52448/0 (socket says 172.21.15.148:52448) 2024-03-04T12:11:48.089 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 -- 172.21.15.148:0/3478709970 learned_addr learned my addr 172.21.15.148:0/3478709970 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2498640 1 --2- 172.21.15.148:0/3478709970 >> v2:172.21.15.191:3300/0 conn(0x7f43bc0cc220 0x7f43bc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 -- 172.21.15.148:0/3478709970 >> v2:172.21.15.191:3300/0 conn(0x7f43bc0cc220 msgr2=0x7f43bc06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 --2- 172.21.15.148:0/3478709970 >> v2:172.21.15.191:3300/0 conn(0x7f43bc0cc220 0x7f43bc06b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 -- 172.21.15.148:0/3478709970 >> v1:172.21.15.191:6789/0 conn(0x7f43bc0cc8e0 legacy=0x7f43bc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.087+0000 7f43c2c99640 1 -- 172.21.15.148:0/3478709970 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc070240 con 0x7f43bc0cad50 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.088+0000 7f43c2c99640 1 --2- 172.21.15.148:0/3478709970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0cb1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f43bc070f70 tx=0x7f43b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc7d0625165659e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.088+0000 7f43c1496640 1 -- 172.21.15.148:0/3478709970 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b402ca80 con 0x7f43bc0cad50 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.088+0000 7f43c1496640 1 -- 172.21.15.148:0/3478709970 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b402cc20 con 0x7f43bc0cad50 2024-03-04T12:11:48.090 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.088+0000 7f43c1496640 1 -- 172.21.15.148:0/3478709970 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40343f0 con 0x7f43bc0cad50 2024-03-04T12:11:48.094 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 -- 172.21.15.148:0/3478709970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 msgr2=0x7f43bc0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.094 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 --2- 172.21.15.148:0/3478709970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0cb1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f43bc070f70 tx=0x7f43b402bee0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 -- 172.21.15.148:0/3478709970 shutdown_connections 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 --2- 172.21.15.148:0/3478709970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0cb1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 --2- 172.21.15.148:0/3478709970 >> v2:172.21.15.191:3300/0 conn(0x7f43bc0cc220 0x7f43bc06b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 -- 172.21.15.148:0/3478709970 >> 172.21.15.148:0/3478709970 conn(0x7f43bc0b22b0 msgr2=0x7f43bc0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 -- 172.21.15.148:0/3478709970 shutdown_connections 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.089+0000 7f43c46fc640 1 -- 172.21.15.148:0/3478709970 wait complete. 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.090+0000 7f43c46fc640 1 Processor -- start 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.090+0000 7f43c46fc640 1 -- start start 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.090+0000 7f43c46fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43bc0cc220 0x7f43bc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 0x7f43bc154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cd3b0 con 0x7f43bc0cad50 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cd0d0 con 0x7f43bc0cc8e0 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cb6e0 con 0x7f43bc0cc220 2024-03-04T12:11:48.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52450/0 (socket says 172.21.15.148:52450) 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 -- 172.21.15.148:0/457268349 learned_addr learned my addr 172.21.15.148:0/457268349 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2c99640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 0x7f43bc154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c1c97640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43bc0cc220 0x7f43bc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 -- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43bc0cc220 msgr2=0x7f43bc1546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43bc0cc220 0x7f43bc1546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 -- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 msgr2=0x7f43bc154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 0x7f43bc154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 -- 172.21.15.148:0/457268349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0c9440 con 0x7f43bc0cad50 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2c99640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 0x7f43bc154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c2498640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d800 tx=0x7f43b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000b6c0 con 0x7f43bc0cad50 2024-03-04T12:11:48.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0019070 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc0c9160 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000ba80 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.091+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc0c5d90 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.092+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc04f090 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.093+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f43b000bc20 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.093+0000 7f43aeffd640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 0x7f43a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.093+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43b008a340 con 0x7f43bc0cad50 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.093+0000 7f43c1c97640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 0x7f43a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.094+0000 7f43c1c97640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 0x7f43a006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43bc06b000 tx=0x7f43b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.098 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.098+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43b0059510 con 0x7f43bc0cad50 2024-03-04T12:11:48.271 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.270+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43bc06ad50 con 0x7f43bc0cad50 2024-03-04T12:11:48.271 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.271+0000 7f43aeffd640 1 -- 172.21.15.148:0/457268349 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7f43b0058e10 con 0x7f43bc0cad50 2024-03-04T12:11:48.272 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-04T12:11:48.272 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":16,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:44.842250+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.272+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 msgr2=0x7f43a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.272+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 0x7f43a006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43bc06b000 tx=0x7f43b8009040 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.272+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 msgr2=0x7f43bc0b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.272+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d800 tx=0x7f43b000dcc0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 shutdown_connections 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f43a006b4d0 0x7f43a006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43bc0cc8e0 0x7f43bc154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43bc0cc220 0x7f43bc1546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 --2- 172.21.15.148:0/457268349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc0cad50 0x7f43bc0b5d10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.274 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 >> 172.21.15.148:0/457268349 conn(0x7f43bc0b22b0 msgr2=0x7f43bc0737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.275 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 shutdown_connections 2024-03-04T12:11:48.275 INFO:teuthology.misc.health.smithi148.stderr:2024-03-04T12:11:48.273+0000 7f43c46fc640 1 -- 172.21.15.148:0/457268349 wait complete. 2024-03-04T12:11:48.283 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-04T12:11:48.284 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-04T12:11:48.284 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T12:11:48.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T12:11:48.286 INFO:tasks.ceph.mds.a:Started 2024-03-04T12:11:48.286 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T12:11:48.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T12:11:48.288 INFO:tasks.ceph.mds.c:Started 2024-03-04T12:11:48.289 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T12:11:48.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T12:11:48.291 INFO:tasks.ceph.mds.b:Started 2024-03-04T12:11:48.292 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-04T12:11:48.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-04T12:11:48.294 INFO:tasks.ceph.mds.d:Started 2024-03-04T12:11:48.294 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-04T12:11:48.294 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:48.362 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-04T12:11:48.365 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-03-04T12:11:48.377 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-04T12:11:48.377 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-04T12:11:48.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.400+0000 7f125cb74640 1 Processor -- start 2024-03-04T12:11:48.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.400+0000 7f125cb74640 1 -- start start 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.401+0000 7f125cb74640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.401+0000 7f125cb74640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.401+0000 7f125cb74640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125815e1e0 con 0x7f12580b70d0 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.401+0000 7f125cb74640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12580d2820 con 0x7f125815de00 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.401+0000 7f125cb74640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12580d1c10 con 0x7f12580b9380 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f1255d9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f125659c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f125659c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:46414/0 (socket says 172.21.15.148:46414) 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f125659c640 1 -- 172.21.15.148:0/2692107733 learned_addr learned my addr 172.21.15.148:0/2692107733 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f1255d9b640 1 -- 172.21.15.148:0/2692107733 >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 msgr2=0x7f12580b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f1255d9b640 1 --2- 172.21.15.148:0/2692107733 >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f1255d9b640 1 -- 172.21.15.148:0/2692107733 >> v1:172.21.15.191:6789/0 conn(0x7f12580b70d0 legacy=0x7f12580b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f1255d9b640 1 -- 172.21.15.148:0/2692107733 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580d1ef0 con 0x7f12580b9380 2024-03-04T12:11:48.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.402+0000 7f125659c640 1 --2- 172.21.15.148:0/2692107733 >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f1255d9b640 1 --2- 172.21.15.148:0/2692107733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580b6b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f124c0097a0 tx=0x7f124c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b0a3303890de6e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125559a640 1 -- 172.21.15.148:0/2692107733 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c035020 con 0x7f12580b9380 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125559a640 1 -- 172.21.15.148:0/2692107733 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c031830 con 0x7f12580b9380 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125559a640 1 -- 172.21.15.148:0/2692107733 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c031c00 con 0x7f12580b9380 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125cb74640 1 -- 172.21.15.148:0/2692107733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 msgr2=0x7f12580b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125cb74640 1 --2- 172.21.15.148:0/2692107733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580b6b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f124c0097a0 tx=0x7f124c02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125cb74640 1 -- 172.21.15.148:0/2692107733 shutdown_connections 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125cb74640 1 --2- 172.21.15.148:0/2692107733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580b6b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.403+0000 7f125cb74640 1 --2- 172.21.15.148:0/2692107733 >> v2:172.21.15.191:3300/0 conn(0x7f125815de00 0x7f12580b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.404+0000 7f125cb74640 1 -- 172.21.15.148:0/2692107733 >> 172.21.15.148:0/2692107733 conn(0x7f12580d0470 msgr2=0x7f12580d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.404+0000 7f125cb74640 1 -- 172.21.15.148:0/2692107733 shutdown_connections 2024-03-04T12:11:48.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.404+0000 7f125cb74640 1 -- 172.21.15.148:0/2692107733 wait complete. 2024-03-04T12:11:48.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.405+0000 7f125cb74640 1 Processor -- start 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.405+0000 7f125cb74640 1 -- start start 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125815de00 0x7f1258153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580b97e0 con 0x7f12580b70d0 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580d24b0 con 0x7f125815de00 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12580b5b80 con 0x7f12580b9380 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52472/0 (socket says 172.21.15.148:52472) 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 -- 172.21.15.148:0/1419085526 learned_addr learned my addr 172.21.15.148:0/1419085526 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f1255d9b640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f1256d9d640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125815de00 0x7f1258153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 -- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 msgr2=0x7f12580c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 -- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125815de00 msgr2=0x7f1258153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125815de00 0x7f1258153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 -- 172.21.15.148:0/1419085526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258152ad0 con 0x7f12580b70d0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125659c640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f124800e9b0 tx=0x7f124800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248017070 con 0x7f12580b70d0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124800c9a0 con 0x7f12580b70d0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1258152db0 con 0x7f12580b70d0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124800cd50 con 0x7f12580b70d0 2024-03-04T12:11:48.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.406+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1258153650 con 0x7f12580b70d0 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.407+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1258152f80 con 0x7f12580b70d0 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.408+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f124801a070 con 0x7f12580b70d0 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.408+0000 7f1246ffd640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.408+0000 7f1255d9b640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.408+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f124808a100 con 0x7f12580b70d0 2024-03-04T12:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.409+0000 7f1255d9b640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 0x7f123806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f124c036000 tx=0x7f124c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.413+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12480592d0 con 0x7f12580b70d0 2024-03-04T12:11:48.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.584+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1258067530 con 0x7f12580b70d0 2024-03-04T12:11:48.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.584+0000 7f1246ffd640 1 -- 172.21.15.148:0/1419085526 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7f1258067530 con 0x7f12580b70d0 2024-03-04T12:11:48.586 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:48.586 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:44.842250+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 msgr2=0x7f123806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 0x7f123806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f124c036000 tx=0x7f124c008030 comp rx=0 tx=0).stop 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 msgr2=0x7f125814e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f124800e9b0 tx=0x7f124800ee70 comp rx=0 tx=0).stop 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 shutdown_connections 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125815de00 0x7f1258153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12580b9380 0x7f12580c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 --2- 172.21.15.148:0/1419085526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12580b70d0 0x7f125814e1c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.586+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 >> 172.21.15.148:0/1419085526 conn(0x7f12580d0470 msgr2=0x7f12580b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.587+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 shutdown_connections 2024-03-04T12:11:48.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.587+0000 7f125cb74640 1 -- 172.21.15.148:0/1419085526 wait complete. 2024-03-04T12:11:48.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T12:11:48.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:11:48.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.701+0000 7f2ab0434640 1 Processor -- start 2024-03-04T12:11:48.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.701+0000 7f2ab0434640 1 -- start start 2024-03-04T12:11:48.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2ab0434640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2aa80b5700 0x7f2aa80d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2ab0434640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2aa806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2ab0434640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa80d2400 con 0x7f2aa80d34a0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2ab0434640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa80d26e0 con 0x7f2aa80b5700 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2ab0434640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80d29c0 con 0x7f2aa80b9410 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad9cf640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2aa80d34a0 0x7f2aa806c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47020/0 (socket says 172.21.15.148:47020) 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad9cf640 1 -- 172.21.15.148:0/617886166 learned_addr learned my addr 172.21.15.148:0/617886166 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926021560 0 0) 0x7f2aa80d2400 con 0x7f2aa80d34a0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aae9d1640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2aa806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a94003520 con 0x7f2aa80d34a0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aae1d0640 1 --2- 172.21.15.148:0/617886166 >> v2:172.21.15.191:3300/0 conn(0x7f2aa80b5700 0x7f2aa80d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163725280 0 0) 0x7f2a94003520 con 0x7f2aa80d34a0 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 msgr2=0x7f2aa806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2aa806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 >> v2:172.21.15.191:3300/0 conn(0x7f2aa80b5700 msgr2=0x7f2aa80d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 --2- 172.21.15.148:0/617886166 >> v2:172.21.15.191:3300/0 conn(0x7f2aa80b5700 0x7f2aa80d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80d2ca0 con 0x7f2aa80d34a0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f2a98001670 con 0x7f2aa80d34a0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 >> v1:172.21.15.148:6789/0 conn(0x7f2aa80d34a0 legacy=0x7f2aa806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a940039b0 0x7f2a94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2a940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80d26e0 con 0x7f2a94004800 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80d29c0 con 0x7f2a940039b0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9400b9c0 con 0x7f2aa80b9410 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aae1d0640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a940039b0 0x7f2a94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.702+0000 7f2aae9d1640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2a940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 -- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 msgr2=0x7f2a940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80b9410 0x7f2a940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 -- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a940039b0 msgr2=0x7f2a94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a940039b0 0x7f2a94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 -- 172.21.15.148:0/617886166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94003520 con 0x7f2a94004800 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2a94004c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a98007cf0 tx=0x7f2a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2908c1c2d7af371c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98007d30 con 0x7f2a94004800 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98005af0 con 0x7f2a94004800 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.703+0000 7f2aad1ce640 1 -- 172.21.15.148:0/617886166 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98007070 con 0x7f2a94004800 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 -- 172.21.15.148:0/617886166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 msgr2=0x7f2a94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 --2- 172.21.15.148:0/617886166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2a94004c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a98007cf0 tx=0x7f2a94005830 comp rx=0 tx=0).stop 2024-03-04T12:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2aae9d1640 1 -- 172.21.15.148:0/617886166 reap_dead start 2024-03-04T12:11:48.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 -- 172.21.15.148:0/617886166 shutdown_connections 2024-03-04T12:11:48.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 -- 172.21.15.148:0/617886166 >> 172.21.15.148:0/617886166 conn(0x7f2aa80d0490 msgr2=0x7f2aa80d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 -- 172.21.15.148:0/617886166 shutdown_connections 2024-03-04T12:11:48.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.704+0000 7f2ab0434640 1 -- 172.21.15.148:0/617886166 wait complete. 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.705+0000 7f2ab0434640 1 Processor -- start 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.705+0000 7f2ab0434640 1 -- start start 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa80cba50 0x7f2aa80ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80cd000 0x7f2aa80cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80b5c60 con 0x7f2a94004800 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80b5ae0 con 0x7f2aa80cba50 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2ab0434640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80b5de0 con 0x7f2aa80cd000 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52498/0 (socket says 172.21.15.148:52498) 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 -- 172.21.15.148:0/2654881804 learned_addr learned my addr 172.21.15.148:0/2654881804 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae9d1640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80cd000 0x7f2aa80cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa80cba50 0x7f2aa80ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 -- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80cd000 msgr2=0x7f2aa80cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80cd000 0x7f2aa80cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 -- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa80cba50 msgr2=0x7f2aa80ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa80cba50 0x7f2aa80ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 -- 172.21.15.148:0/2654881804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa81ef270 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.706+0000 7f2aae1d0640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40118e0 tx=0x7f2aa4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.707+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400faf0 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.707+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400fc90 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.707+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa80cc290 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.707+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4014440 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.707+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa81ef800 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.708+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa8074f60 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.708+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2aa4012040 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.708+0000 7f2a9effd640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 0x7f2a8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.709+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2aa408a550 con 0x7f2a94004800 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.709+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 0x7f2a8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:48.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.709+0000 7f2aad9cf640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 0x7f2a8c06d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a9803c000 tx=0x7f2a98009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:48.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.713+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2aa4019030 con 0x7f2a94004800 2024-03-04T12:11:48.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.888+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2aa80b6240 con 0x7f2a94004800 2024-03-04T12:11:48.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.888+0000 7f2a9effd640 1 -- 172.21.15.148:0/2654881804 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2aa40590d0 con 0x7f2a94004800 2024-03-04T12:11:48.889 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:11:48.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 msgr2=0x7f2a8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 0x7f2a8c06d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a9803c000 tx=0x7f2a98009030 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 msgr2=0x7f2aa80cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40118e0 tx=0x7f2aa4011da0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 shutdown_connections 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2a8c06b500 0x7f2a8c06d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2aa80cd000 0x7f2aa80cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa80cba50 0x7f2aa80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 --2- 172.21.15.148:0/2654881804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a94004800 0x7f2aa80cb510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 >> 172.21.15.148:0/2654881804 conn(0x7f2aa80d0490 msgr2=0x7f2aa80c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 shutdown_connections 2024-03-04T12:11:48.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:48.890+0000 7f2ab0434640 1 -- 172.21.15.148:0/2654881804 wait complete. 2024-03-04T12:11:48.903 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:11:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.002+0000 7f363fd9e640 1 Processor -- start 2024-03-04T12:11:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.002+0000 7f363fd9e640 1 -- start start 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363fd9e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f36380b0700 0x7f36380b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363fd9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36380ccad0 con 0x7f363815b960 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363fd9e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36380cfab0 con 0x7f36380b47c0 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363fd9e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36380cf860 con 0x7f36380b0700 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363db3a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f363815b960 0x7f36380d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47024/0 (socket says 172.21.15.148:47024) 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363db3a640 1 -- 172.21.15.148:0/3084139377 learned_addr learned my addr 172.21.15.148:0/3084139377 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.003+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548308941 0 0) 0x7f36380ccad0 con 0x7f363815b960 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3624003520 con 0x7f363815b960 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363d339640 1 --2- 172.21.15.148:0/3084139377 >> v2:172.21.15.191:3300/0 conn(0x7f36380b0700 0x7f36380b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400257069 0 0) 0x7f3624003520 con 0x7f363815b960 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 >> v2:172.21.15.191:3300/0 conn(0x7f36380b0700 msgr2=0x7f36380b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 --2- 172.21.15.148:0/3084139377 >> v2:172.21.15.191:3300/0 conn(0x7f36380b0700 0x7f36380b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 >> v1:172.21.15.191:6789/0 conn(0x7f36380b47c0 legacy=0x7f36380cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380cfd90 con 0x7f363815b960 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f36340019f0 con 0x7f363815b960 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 >> v1:172.21.15.148:6789/0 conn(0x7f363815b960 legacy=0x7f36380d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380cf860 con 0x7f36240039b0 2024-03-04T12:11:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362400c160 con 0x7f36240051c0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362400c360 con 0x7f3624004800 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363d339640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363e33b640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 -- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3624004800 msgr2=0x7f3624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 -- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240051c0 msgr2=0x7f362400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363db3a640 1 -- 172.21.15.148:0/3084139377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624003520 con 0x7f36240039b0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363e33b640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.004+0000 7f363e33b640 1 -- 172.21.15.148:0/3084139377 reap_dead start 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.005+0000 7f363db3a640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f363400a840 tx=0x7f3624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5702be038f57c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.005+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400a460 con 0x7f36240039b0 2024-03-04T12:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.005+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634034650 con 0x7f36240039b0 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.005+0000 7f363cb38640 1 -- 172.21.15.148:0/3084139377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634034a40 con 0x7f36240039b0 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 -- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 msgr2=0x7f3624003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f363400a840 tx=0x7f3624005830 comp rx=0 tx=0).stop 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 -- 172.21.15.148:0/3084139377 shutdown_connections 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 --2- 172.21.15.148:0/3084139377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 -- 172.21.15.148:0/3084139377 >> 172.21.15.148:0/3084139377 conn(0x7f363806f5f0 msgr2=0x7f363806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 -- 172.21.15.148:0/3084139377 shutdown_connections 2024-03-04T12:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.006+0000 7f363fd9e640 1 -- 172.21.15.148:0/3084139377 wait complete. 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.007+0000 7f363fd9e640 1 Processor -- start 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.007+0000 7f363fd9e640 1 -- start start 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.007+0000 7f363fd9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240039b0 0x7f36381565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.007+0000 7f363fd9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 0x7f3638158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b0b90 con 0x7f363815e1d0 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380d0100 con 0x7f36240039b0 2024-03-04T12:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b0d10 con 0x7f36381556f0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52522/0 (socket says 172.21.15.148:52522) 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 -- 172.21.15.148:0/2831230023 learned_addr learned my addr 172.21.15.148:0/2831230023 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363db3a640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240039b0 0x7f36381565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363d339640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 0x7f3638158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 -- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 msgr2=0x7f3638158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 0x7f3638158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 -- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240039b0 msgr2=0x7f36381565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240039b0 0x7f36381565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 -- 172.21.15.148:0/2831230023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638156fa0 con 0x7f363815e1d0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363d339640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 0x7f3638158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363e33b640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3638067960 tx=0x7f363000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630019070 con 0x7f363815e1d0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36300050d0 con 0x7f363815e1d0 2024-03-04T12:11:49.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300035f0 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3638157260 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.008+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3638157820 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.009+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3638074f60 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.010+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f363001c070 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.010+0000 7f362e7fc640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 0x7f361c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.010+0000 7f363db3a640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 0x7f361c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.010+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3630089100 con 0x7f363815e1d0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.010+0000 7f363db3a640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 0x7f361c06d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3634034000 tx=0x7f3634009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.014+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3630057c80 con 0x7f363815e1d0 2024-03-04T12:11:49.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.219+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36380cd180 con 0x7f363815e1d0 2024-03-04T12:11:49.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.219+0000 7f362e7fc640 1 -- 172.21.15.148:0/2831230023 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f363005bd20 con 0x7f363815e1d0 2024-03-04T12:11:49.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:49.220 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T12:11:49.221 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 2 2024-03-04T12:11:49.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 msgr2=0x7f361c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 0x7f361c06d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3634034000 tx=0x7f3634009030 comp rx=0 tx=0).stop 2024-03-04T12:11:49.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 msgr2=0x7f3638154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3638067960 tx=0x7f363000e820 comp rx=0 tx=0).stop 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 shutdown_connections 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f361c06b500 0x7f361c06d9c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f363815e1d0 0x7f3638154830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36381556f0 0x7f3638158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 --2- 172.21.15.148:0/2831230023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36240039b0 0x7f36381565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 >> 172.21.15.148:0/2831230023 conn(0x7f363806f5f0 msgr2=0x7f36380b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 shutdown_connections 2024-03-04T12:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:49.221+0000 7f363fd9e640 1 -- 172.21.15.148:0/2831230023 wait complete. 2024-03-04T12:11:51.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:11:51.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.344+0000 7fcfafdb9640 1 Processor -- start 2024-03-04T12:11:51.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.344+0000 7fcfafdb9640 1 -- start start 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfafdb9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfa8154a80 0x7fcfa815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfafdb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa8161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfafdb9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa80cff70 con 0x7fcfa80cf650 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfafdb9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa8071df0 con 0x7fcfa8154a80 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfafdb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80cd010 con 0x7fcfa80cfae0 2024-03-04T12:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfadb55640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfa8154a80 0x7fcfa815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfad354640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcfa80cf650 0x7fcfa81616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47040/0 (socket says 172.21.15.148:47040) 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfad354640 1 -- 172.21.15.148:0/2518737746 learned_addr learned my addr 172.21.15.148:0/2518737746 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfae356640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa8161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfadb55640 1 -- 172.21.15.148:0/2518737746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 msgr2=0x7fcfa8161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfadb55640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa8161dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfadb55640 1 -- 172.21.15.148:0/2518737746 >> v1:172.21.15.148:6789/0 conn(0x7fcfa80cf650 legacy=0x7fcfa81616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfacb53640 1 -- 172.21.15.148:0/2518737746 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096503413 0 0) 0x7fcfa80cff70 con 0x7fcfa80cf650 2024-03-04T12:11:51.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.345+0000 7fcfadb55640 1 -- 172.21.15.148:0/2518737746 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80cdaa0 con 0x7fcfa8154a80 2024-03-04T12:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfadb55640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa815e910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097a0 tx=0x7fcfa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cf3ac286f928ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfacb53640 1 -- 172.21.15.148:0/2518737746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4035020 con 0x7fcfa8154a80 2024-03-04T12:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfacb53640 1 -- 172.21.15.148:0/2518737746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4031830 con 0x7fcfa8154a80 2024-03-04T12:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfacb53640 1 -- 172.21.15.148:0/2518737746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4031c00 con 0x7fcfa8154a80 2024-03-04T12:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 -- 172.21.15.148:0/2518737746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 msgr2=0x7fcfa815e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa815e910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40097a0 tx=0x7fcfa402b8d0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 -- 172.21.15.148:0/2518737746 shutdown_connections 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa8161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/2518737746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa815e910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.346+0000 7fcfafdb9640 1 -- 172.21.15.148:0/2518737746 >> 172.21.15.148:0/2518737746 conn(0x7fcfa806f5f0 msgr2=0x7fcfa806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.347+0000 7fcfafdb9640 1 -- 172.21.15.148:0/2518737746 shutdown_connections 2024-03-04T12:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.347+0000 7fcfafdb9640 1 -- 172.21.15.148:0/2518737746 wait complete. 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.348+0000 7fcfafdb9640 1 Processor -- start 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.348+0000 7fcfafdb9640 1 -- start start 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.348+0000 7fcfafdb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 0x7fcfa80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.348+0000 7fcfafdb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa81499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.348+0000 7fcfafdb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfafdb9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80ce340 con 0x7fcfa8154a80 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfafdb9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80ce060 con 0x7fcfa80cf650 2024-03-04T12:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfafdb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80ce620 con 0x7fcfa80cfae0 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52534/0 (socket says 172.21.15.148:52534) 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 -- 172.21.15.148:0/3296534596 learned_addr learned my addr 172.21.15.148:0/3296534596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfad354640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa81499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfadb55640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 0x7fcfa80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 -- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 msgr2=0x7fcfa81499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa81499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 -- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 msgr2=0x7fcfa80c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 0x7fcfa80c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 -- 172.21.15.148:0/3296534596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8150540 con 0x7fcfa8154a80 2024-03-04T12:11:51.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfadb55640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 0x7fcfa80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfae356640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000cda0 tx=0x7fcfa000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0003b90 con 0x7fcfa8154a80 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0003d30 con 0x7fcfa8154a80 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa80c7d90 con 0x7fcfa8154a80 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0004240 con 0x7fcfa8154a80 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.349+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa80c8070 con 0x7fcfa8154a80 2024-03-04T12:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.350+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa8074f60 con 0x7fcfa8154a80 2024-03-04T12:11:51.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.351+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcfa00043e0 con 0x7fcfa8154a80 2024-03-04T12:11:51.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.351+0000 7fcf9e7fc640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 0x7fcf8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.351+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfa008a990 con 0x7fcfa8154a80 2024-03-04T12:11:51.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.351+0000 7fcfadb55640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 0x7fcf8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.352+0000 7fcfadb55640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 0x7fcf8c06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4036000 tx=0x7fcfa4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.355+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfa0059b60 con 0x7fcfa8154a80 2024-03-04T12:11:51.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.561+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfa806c030 con 0x7fcfa8154a80 2024-03-04T12:11:51.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.561+0000 7fcf9e7fc640 1 -- 172.21.15.148:0/3296534596 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7fcfa0059460 con 0x7fcfa8154a80 2024-03-04T12:11:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:51.563 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-04T12:11:51.563 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 3 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 msgr2=0x7fcf8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 0x7fcf8c06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4036000 tx=0x7fcfa4008030 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 msgr2=0x7fcfa80c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000cda0 tx=0x7fcfa000a660 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 shutdown_connections 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8c06b4d0 0x7fcf8c06d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa8154a80 0x7fcfa80c54e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa80cfae0 0x7fcfa81499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 --2- 172.21.15.148:0/3296534596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa80cf650 0x7fcfa80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 >> 172.21.15.148:0/3296534596 conn(0x7fcfa806f5f0 msgr2=0x7fcfa806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 shutdown_connections 2024-03-04T12:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.563+0000 7fcfafdb9640 1 -- 172.21.15.148:0/3296534596 wait complete. 2024-03-04T12:11:51.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:11:51.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.675+0000 7fc923668640 1 Processor -- start 2024-03-04T12:11:51.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.676+0000 7fc923668640 1 -- start start 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.676+0000 7fc923668640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc91c154a80 0x7fc91c15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.676+0000 7fc923668640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c1624e0 con 0x7fc91c0cfae0 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.676+0000 7fc923668640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0cd010 con 0x7fc91c0cf650 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.676+0000 7fc923668640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c071df0 con 0x7fc91c154a80 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921c05640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc91c0cfae0 0x7fc91c161dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47044/0 (socket says 172.21.15.148:47044) 2024-03-04T12:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921c05640 1 -- 172.21.15.148:0/1418385863 learned_addr learned my addr 172.21.15.148:0/1418385863 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921404640 1 --2- 172.21.15.148:0/1418385863 >> v2:172.21.15.148:3300/0 conn(0x7fc91c154a80 0x7fc91c15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225505929 0 0) 0x7fc91c1624e0 con 0x7fc91c0cfae0 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc908003520 con 0x7fc91c0cfae0 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246861208 0 0) 0x7fc91c0cd010 con 0x7fc91c0cf650 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921404640 1 -- 172.21.15.148:0/1418385863 >> v1:172.21.15.191:6789/0 conn(0x7fc91c0cf650 legacy=0x7fc91c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921404640 1 -- 172.21.15.148:0/1418385863 >> v1:172.21.15.148:6789/0 conn(0x7fc91c0cfae0 legacy=0x7fc91c161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.677+0000 7fc921404640 1 -- 172.21.15.148:0/1418385863 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0cff70 con 0x7fc91c154a80 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.678+0000 7fc921404640 1 --2- 172.21.15.148:0/1418385863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c154a80 0x7fc91c15e910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82c814303067711 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.678+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c034070 con 0x7fc91c154a80 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.678+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c030610 con 0x7fc91c154a80 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.678+0000 7fc913fff640 1 -- 172.21.15.148:0/1418385863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0309e0 con 0x7fc91c154a80 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 -- 172.21.15.148:0/1418385863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c154a80 msgr2=0x7fc91c15e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 --2- 172.21.15.148:0/1418385863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c154a80 0x7fc91c15e910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c02b960 comp rx=0 tx=0).stop 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 -- 172.21.15.148:0/1418385863 shutdown_connections 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 --2- 172.21.15.148:0/1418385863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c154a80 0x7fc91c15e910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 -- 172.21.15.148:0/1418385863 >> 172.21.15.148:0/1418385863 conn(0x7fc91c06f5f0 msgr2=0x7fc91c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 -- 172.21.15.148:0/1418385863 shutdown_connections 2024-03-04T12:11:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.679+0000 7fc923668640 1 -- 172.21.15.148:0/1418385863 wait complete. 2024-03-04T12:11:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.680+0000 7fc923668640 1 Processor -- start 2024-03-04T12:11:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.680+0000 7fc923668640 1 -- start start 2024-03-04T12:11:51.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 0x7fc91c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 0x7fc91c1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0ce060 con 0x7fc91c0cfae0 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0cdd80 con 0x7fc91c0cf650 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0ce340 con 0x7fc91c154a80 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52560/0 (socket says 172.21.15.148:52560) 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 -- 172.21.15.148:0/3167105600 learned_addr learned my addr 172.21.15.148:0/3167105600 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc921c05640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 0x7fc91c1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc921404640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 0x7fc91c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 -- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 msgr2=0x7fc91c1f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 0x7fc91c1f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 -- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 msgr2=0x7fc91c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 0x7fc91c150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 -- 172.21.15.148:0/3167105600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0c8e80 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc921404640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 0x7fc91c150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc921c05640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 0x7fc91c1f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc920c03640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c4b0 tx=0x7fc91800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918017070 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918005480 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c151490 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.681+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918003830 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.682+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c151a50 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.683+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c074f60 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.683+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc91801a070 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.683+0000 7fc911ffb640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 0x7fc90006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.683+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9180891d0 con 0x7fc91c0cfae0 2024-03-04T12:11:51.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.683+0000 7fc921404640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 0x7fc90006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.684+0000 7fc921404640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 0x7fc90006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.687+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc918058430 con 0x7fc91c0cfae0 2024-03-04T12:11:51.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.858+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc91c1537c0 con 0x7fc91c0cfae0 2024-03-04T12:11:51.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.859+0000 7fc911ffb640 1 -- 172.21.15.148:0/3167105600 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7fc918057d30 con 0x7fc91c0cfae0 2024-03-04T12:11:51.860 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:51.860 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:44.842250+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 msgr2=0x7fc90006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 0x7fc90006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c008030 comp rx=0 tx=0).stop 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 msgr2=0x7fc91c0c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c4b0 tx=0x7fc91800c970 comp rx=0 tx=0).stop 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 shutdown_connections 2024-03-04T12:11:51.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc90006b4d0 0x7fc90006d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc91c154a80 0x7fc91c1f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc91c0cfae0 0x7fc91c0c8940 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 --2- 172.21.15.148:0/3167105600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91c0cf650 0x7fc91c150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 >> 172.21.15.148:0/3167105600 conn(0x7fc91c06f5f0 msgr2=0x7fc91c1528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 shutdown_connections 2024-03-04T12:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.862+0000 7fc923668640 1 -- 172.21.15.148:0/3167105600 wait complete. 2024-03-04T12:11:51.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T12:11:51.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:11:51.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.976+0000 7f0b1c044640 1 Processor -- start 2024-03-04T12:11:51.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.976+0000 7f0b1c044640 1 -- start start 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1c044640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1415da20 0x7f0b140ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1c044640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b14161df0 0x7f0b140cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1c044640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b14162450 con 0x7f0b140cf660 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1c044640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b140d0000 con 0x7f0b14161df0 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1c044640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cc920 con 0x7f0b1415da20 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b14161df0 0x7f0b140cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b14161df0 0x7f0b140cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52564/0 (socket says 172.21.15.148:52564) 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/375374787 learned_addr learned my addr 172.21.15.148:0/375374787 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b19de0640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1415da20 0x7f0b140ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/375374787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1415da20 msgr2=0x7f0b140ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1415da20 0x7f0b140ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.977+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/375374787 >> v1:172.21.15.191:6789/0 conn(0x7f0b140cf660 legacy=0x7f0b141616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.978+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/375374787 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140cc2d0 con 0x7f0b14161df0 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.978+0000 7f0b1a5e1640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140cfac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8933b4791e77346 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.978+0000 7f0b18dde640 1 -- 172.21.15.148:0/375374787 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c034070 con 0x7f0b14161df0 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.978+0000 7f0b18dde640 1 -- 172.21.15.148:0/375374787 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c030840 con 0x7f0b14161df0 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b18dde640 1 -- 172.21.15.148:0/375374787 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c030c10 con 0x7f0b14161df0 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 -- 172.21.15.148:0/375374787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 msgr2=0x7f0b140cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140cfac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009870 tx=0x7f0b0c02b980 comp rx=0 tx=0).stop 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 -- 172.21.15.148:0/375374787 shutdown_connections 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140cfac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 --2- 172.21.15.148:0/375374787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1415da20 0x7f0b140ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 -- 172.21.15.148:0/375374787 >> 172.21.15.148:0/375374787 conn(0x7f0b140b33a0 msgr2=0x7f0b140b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 -- 172.21.15.148:0/375374787 shutdown_connections 2024-03-04T12:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.979+0000 7f0b1c044640 1 -- 172.21.15.148:0/375374787 wait complete. 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.980+0000 7f0b1c044640 1 Processor -- start 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.980+0000 7f0b1c044640 1 -- start start 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 0x7f0b140b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 0x7f0b140b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cbf80 con 0x7f0b14161df0 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cd2f0 con 0x7f0b1415da20 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1c044640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1415de00 con 0x7f0b140cf660 2024-03-04T12:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52566/0 (socket says 172.21.15.148:52566) 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/426569663 learned_addr learned my addr 172.21.15.148:0/426569663 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b19de0640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 0x7f0b140b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b195df640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 0x7f0b140b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 msgr2=0x7f0b140b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 0x7f0b140b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 msgr2=0x7f0b140b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 0x7f0b140b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b1a5e1640 1 -- 172.21.15.148:0/426569663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140cb130 con 0x7f0b14161df0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b19de0640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 0x7f0b140b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.981+0000 7f0b195df640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 0x7f0b140b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b1a5e1640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c02b9c0 tx=0x7f0b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c034040 con 0x7f0b14161df0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c03ada0 con 0x7f0b14161df0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b140cb410 con 0x7f0b14161df0 2024-03-04T12:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c044630 con 0x7f0b14161df0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.982+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140c53c0 con 0x7f0b14161df0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.983+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b14074f60 con 0x7f0b14161df0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.983+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0b0c037070 con 0x7f0b14161df0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.983+0000 7f0b0a7fc640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.983+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b0c0ac050 con 0x7f0b14161df0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.984+0000 7f0b19de0640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:51.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.984+0000 7f0b19de0640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 0x7f0af806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b141f6fd0 tx=0x7f0b100039f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:51.988+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b0c03f070 con 0x7f0b14161df0 2024-03-04T12:11:52.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.158+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b14161a00 con 0x7f0b14161df0 2024-03-04T12:11:52.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.159+0000 7f0b0a7fc640 1 -- 172.21.15.148:0/426569663 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b0c07b260 con 0x7f0b14161df0 2024-03-04T12:11:52.161 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.160+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 msgr2=0x7f0af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.160+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 0x7f0af806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b141f6fd0 tx=0x7f0b100039f0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.160+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 msgr2=0x7f0b140ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.160+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c02b9c0 tx=0x7f0b0c0046c0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 shutdown_connections 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b14161df0 0x7f0b140ca370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1415da20 0x7f0b140b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 --2- 172.21.15.148:0/426569663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b140cf660 0x7f0b140b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 >> 172.21.15.148:0/426569663 conn(0x7f0b140b33a0 msgr2=0x7f0b1406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 shutdown_connections 2024-03-04T12:11:52.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.161+0000 7f0b1c044640 1 -- 172.21.15.148:0/426569663 wait complete. 2024-03-04T12:11:52.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-04T12:11:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.271+0000 7fa3600cd640 1 Processor -- start 2024-03-04T12:11:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.271+0000 7fa3600cd640 1 -- start start 2024-03-04T12:11:52.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa35815dd70 0x7fa3580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3580b9710 0x7fa3580ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa35806c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580cf200 con 0x7fa3580b9710 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35815e150 con 0x7fa35815dd70 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa3600cd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580d2630 con 0x7fa3580ced90 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa35d668640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3580b9710 0x7fa3580ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa35d668640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3580b9710 0x7fa3580ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52580/0 (socket says 172.21.15.148:52580) 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa35d668640 1 -- 172.21.15.148:0/1275608841 learned_addr learned my addr 172.21.15.148:0/1275608841 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa35e66a640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa35806c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.272+0000 7fa35de69640 1 --2- 172.21.15.148:0/1275608841 >> v2:172.21.15.191:3300/0 conn(0x7fa35815dd70 0x7fa3580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 -- 172.21.15.148:0/1275608841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 msgr2=0x7fa35806c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa35806c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 -- 172.21.15.148:0/1275608841 >> v2:172.21.15.191:3300/0 conn(0x7fa35815dd70 msgr2=0x7fa3580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 --2- 172.21.15.148:0/1275608841 >> v2:172.21.15.191:3300/0 conn(0x7fa35815dd70 0x7fa3580d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 -- 172.21.15.148:0/1275608841 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580d2910 con 0x7fa3580b9710 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35d668640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3580ce850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3580d18c0 tx=0x7fa34802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84efcd8ba38220ba server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.273+0000 7fa35ce67640 1 -- 172.21.15.148:0/1275608841 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348035020 con 0x7fa3580b9710 2024-03-04T12:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa35ce67640 1 -- 172.21.15.148:0/1275608841 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348031870 con 0x7fa3580b9710 2024-03-04T12:11:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa35ce67640 1 -- 172.21.15.148:0/1275608841 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348031c40 con 0x7fa3580b9710 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 -- 172.21.15.148:0/1275608841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 msgr2=0x7fa3580ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3580ce850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3580d18c0 tx=0x7fa34802b910 comp rx=0 tx=0).stop 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 -- 172.21.15.148:0/1275608841 shutdown_connections 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa35806c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1275608841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3580ce850 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1275608841 >> v2:172.21.15.191:3300/0 conn(0x7fa35815dd70 0x7fa3580d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 -- 172.21.15.148:0/1275608841 >> 172.21.15.148:0/1275608841 conn(0x7fa358072a60 msgr2=0x7fa3580d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 -- 172.21.15.148:0/1275608841 shutdown_connections 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.274+0000 7fa3600cd640 1 -- 172.21.15.148:0/1275608841 wait complete. 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.275+0000 7fa3600cd640 1 Processor -- start 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.275+0000 7fa3600cd640 1 -- start start 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa358152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35815dd70 0x7fa358153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580d31b0 con 0x7fa3580b9710 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580d2ed0 con 0x7fa35815dd70 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa3600cd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580d2320 con 0x7fa3580ced90 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52582/0 (socket says 172.21.15.148:52582) 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 -- 172.21.15.148:0/1546539140 learned_addr learned my addr 172.21.15.148:0/1546539140 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35d668640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa358152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 -- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 msgr2=0x7fa358152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa358152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 -- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35815dd70 msgr2=0x7fa358153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35815dd70 0x7fa358153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 -- 172.21.15.148:0/1546539140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3581f3ee0 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.276+0000 7fa35de69640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa354007ef0 tx=0x7fa35400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354014620 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa354019070 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35815b3e0 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354014a00 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35815b9a0 con 0x7fa3580b9710 2024-03-04T12:11:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.277+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358074f60 con 0x7fa3580b9710 2024-03-04T12:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.278+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa354014ba0 con 0x7fa3580b9710 2024-03-04T12:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.278+0000 7fa34e7fc640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 0x7fa33c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.278+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa35408a7f0 con 0x7fa3580b9710 2024-03-04T12:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.278+0000 7fa35d668640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 0x7fa33c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.279+0000 7fa35d668640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 0x7fa33c06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa348036000 tx=0x7fa348008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.282+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3540599c0 con 0x7fa3580b9710 2024-03-04T12:11:52.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.454+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3580b6b70 con 0x7fa3580b9710 2024-03-04T12:11:52.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.454+0000 7fa34e7fc640 1 -- 172.21.15.148:0/1546539140 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa3540592c0 con 0x7fa3580b9710 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:11:52.456 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 msgr2=0x7fa33c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 0x7fa33c06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa348036000 tx=0x7fa348008030 comp rx=0 tx=0).stop 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 msgr2=0x7fa3581542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa354007ef0 tx=0x7fa35400e4b0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 shutdown_connections 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa33c06b4d0 0x7fa33c06d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35815dd70 0x7fa358153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3580ced90 0x7fa358152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 --2- 172.21.15.148:0/1546539140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3580b9710 0x7fa3581542a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 >> 172.21.15.148:0/1546539140 conn(0x7fa358072a60 msgr2=0x7fa3580ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 shutdown_connections 2024-03-04T12:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.456+0000 7fa3600cd640 1 -- 172.21.15.148:0/1546539140 wait complete. 2024-03-04T12:11:52.468 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-04T12:11:52.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-04T12:11:52.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.572+0000 7f846fb1b640 1 Processor -- start 2024-03-04T12:11:52.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.572+0000 7f846fb1b640 1 -- start start 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846fb1b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f846815dc20 0x7f84680d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846fb1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84680d0800 0x7f84680d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846fb1b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815e000 con 0x7f84680d2ab0 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846fb1b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680d1190 con 0x7f84680d0800 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846fb1b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680d2f50 con 0x7f846815dc20 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846e0b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84680d0800 0x7f84680d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846d0b6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f84680d2ab0 0x7f84680d3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40394/0 (socket says 172.21.15.148:40394) 2024-03-04T12:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846d0b6640 1 -- 172.21.15.148:0/737938144 learned_addr learned my addr 172.21.15.148:0/737938144 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846e0b8640 1 -- 172.21.15.148:0/737938144 >> v2:172.21.15.191:3300/0 conn(0x7f846815dc20 msgr2=0x7f84680d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846d8b7640 1 --2- 172.21.15.148:0/737938144 >> v2:172.21.15.191:3300/0 conn(0x7f846815dc20 0x7f84680d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846e0b8640 1 --2- 172.21.15.148:0/737938144 >> v2:172.21.15.191:3300/0 conn(0x7f846815dc20 0x7f84680d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846e0b8640 1 -- 172.21.15.148:0/737938144 >> v1:172.21.15.148:6789/0 conn(0x7f84680d2ab0 legacy=0x7f84680d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846e0b8640 1 -- 172.21.15.148:0/737938144 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680d3230 con 0x7f84680d0800 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.573+0000 7f846c8b5640 1 -- 172.21.15.148:0/737938144 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873510579 0 0) 0x7f846815e000 con 0x7f84680d2ab0 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846e0b8640 1 --2- 172.21.15.148:0/737938144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84680d0800 0x7f84680d0c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84680b5f80 tx=0x7f846002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f31f9d7b74c3280 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846c8b5640 1 -- 172.21.15.148:0/737938144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460039070 con 0x7f84680d0800 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846c8b5640 1 -- 172.21.15.148:0/737938144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846002cc80 con 0x7f84680d0800 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846c8b5640 1 -- 172.21.15.148:0/737938144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460034660 con 0x7f84680d0800 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 -- 172.21.15.148:0/737938144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84680d0800 msgr2=0x7f84680d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 --2- 172.21.15.148:0/737938144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84680d0800 0x7f84680d0c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84680b5f80 tx=0x7f846002bed0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 -- 172.21.15.148:0/737938144 shutdown_connections 2024-03-04T12:11:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 --2- 172.21.15.148:0/737938144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84680d0800 0x7f84680d0c50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 --2- 172.21.15.148:0/737938144 >> v2:172.21.15.191:3300/0 conn(0x7f846815dc20 0x7f84680d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 -- 172.21.15.148:0/737938144 >> 172.21.15.148:0/737938144 conn(0x7f84680725e0 msgr2=0x7f84680729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.574+0000 7f846fb1b640 1 -- 172.21.15.148:0/737938144 shutdown_connections 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.575+0000 7f846fb1b640 1 -- 172.21.15.148:0/737938144 wait complete. 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 Processor -- start 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 -- start start 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84680d0800 0x7f84680ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 0x7f84680cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680d3ad0 con 0x7f846815dc20 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680d37f0 con 0x7f84680d2ab0 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846fb1b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680d4700 con 0x7f84680d0800 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846e0b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846e0b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41318/0 (socket says 172.21.15.148:41318) 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.576+0000 7f846e0b8640 1 -- 172.21.15.148:0/2699428657 learned_addr learned my addr 172.21.15.148:0/2699428657 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846d8b7640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84680d0800 0x7f84680ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846d0b6640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 0x7f84680cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 -- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84680d0800 msgr2=0x7f84680ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84680d0800 0x7f84680ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 -- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 msgr2=0x7f84680cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 0x7f84680cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 -- 172.21.15.148:0/2699428657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846814dcd0 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846d0b6640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 0x7f84680cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846e0b8640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f846002c4e0 tx=0x7f8460004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460039050 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460045070 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846814e120 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846003e830 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.577+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846814e6a0 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.578+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8468074f60 con 0x7f846815dc20 2024-03-04T12:11:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.578+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f846003e9d0 con 0x7f846815dc20 2024-03-04T12:11:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.579+0000 7f845e7fc640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 0x7f844c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.579+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84600ad010 con 0x7f846815dc20 2024-03-04T12:11:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.579+0000 7f846d8b7640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 0x7f844c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.579+0000 7f846d8b7640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 0x7f844c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.583+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f846007c300 con 0x7f846815dc20 2024-03-04T12:11:52.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.757+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f846815d2a0 con 0x7f846815dc20 2024-03-04T12:11:52.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.866+0000 7f845e7fc640 1 -- 172.21.15.148:0/2699428657 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f846007bc00 con 0x7f846815dc20 2024-03-04T12:11:52.867 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.868+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 msgr2=0x7f844c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 0x7f844c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).stop 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 msgr2=0x7f84680cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f846002c4e0 tx=0x7f8460004a40 comp rx=0 tx=0).stop 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 shutdown_connections 2024-03-04T12:11:52.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f844c06b4d0 0x7f844c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846815dc20 0x7f84680cd380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84680d2ab0 0x7f84680cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 --2- 172.21.15.148:0/2699428657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84680d0800 0x7f84680ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 >> 172.21.15.148:0/2699428657 conn(0x7f84680725e0 msgr2=0x7f8468162080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 shutdown_connections 2024-03-04T12:11:52.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.869+0000 7f846fb1b640 1 -- 172.21.15.148:0/2699428657 wait complete. 2024-03-04T12:11:52.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-04T12:11:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.991+0000 7fbcb92d7640 1 Processor -- start 2024-03-04T12:11:52.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.991+0000 7fbcb92d7640 1 -- start start 2024-03-04T12:11:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb92d7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbcb40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb92d7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb415c790 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb92d7640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb415ca50 con 0x7fbcb415c2e0 2024-03-04T12:11:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb92d7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415d2f0 con 0x7fbcb40ceec0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbcb40ccbd0 0x7fbcb40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40398/0 (socket says 172.21.15.148:40398) 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb2ffd640 1 -- 172.21.15.148:0/540482411 learned_addr learned my addr 172.21.15.148:0/540482411 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.992+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791740914 0 0) 0x7fbcb415c790 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc9c003520 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb27fc640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbcb40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2374144548 0 0) 0x7fbc9c003520 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 msgr2=0x7fbcb40b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbcb40b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 >> v1:172.21.15.191:6789/0 conn(0x7fbcb415c2e0 legacy=0x7fbcb40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb415cd30 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fbca40019f0 con 0x7fbcb40ccbd0 2024-03-04T12:11:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 >> v1:172.21.15.148:6789/0 conn(0x7fbcb40ccbd0 legacy=0x7fbcb40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbc9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c003f60 0x7fbc9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9c004d40 0x7fbc9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c00ba70 con 0x7fbc9c004d40 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c00bc20 con 0x7fbc9c003f60 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c00be20 con 0x7fbcb40ceec0 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9c004d40 0x7fbc9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb2ffd640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbc9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb27fc640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c003f60 0x7fbc9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 -- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 msgr2=0x7fbc9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb40ceec0 0x7fbc9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 -- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c003f60 msgr2=0x7fbc9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c003f60 0x7fbc9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.993+0000 7fbcb37fe640 1 -- 172.21.15.148:0/540482411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c003520 con 0x7fbc9c004d40 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.994+0000 7fbcb37fe640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9c004d40 0x7fbc9c0051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcac001750 tx=0x7fbc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a0623810a163e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.994+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac001370 con 0x7fbc9c004d40 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.994+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac0052d0 con 0x7fbc9c004d40 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.994+0000 7fbcb1ffb640 1 -- 172.21.15.148:0/540482411 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac003830 con 0x7fbc9c004d40 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 -- 172.21.15.148:0/540482411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9c004d40 msgr2=0x7fbc9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/540482411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9c004d40 0x7fbc9c0051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcac001750 tx=0x7fbc9c005830 comp rx=0 tx=0).stop 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb37fe640 1 -- 172.21.15.148:0/540482411 reap_dead start 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 -- 172.21.15.148:0/540482411 shutdown_connections 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 -- 172.21.15.148:0/540482411 >> 172.21.15.148:0/540482411 conn(0x7fbcb4066dc0 msgr2=0x7fbcb40671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 -- 172.21.15.148:0/540482411 shutdown_connections 2024-03-04T12:11:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.995+0000 7fbcb92d7640 1 -- 172.21.15.148:0/540482411 wait complete. 2024-03-04T12:11:53.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.996+0000 7fbcb92d7640 1 Processor -- start 2024-03-04T12:11:53.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.996+0000 7fbcb92d7640 1 -- start start 2024-03-04T12:11:53.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c004d40 0x7fbcb40c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:53.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb406bde0 0x7fbcb40d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40ccfb0 con 0x7fbcb415b460 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415d6d0 con 0x7fbc9c004d40 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb92d7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415eb70 con 0x7fbcb406bde0 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41346/0 (socket says 172.21.15.148:41346) 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 -- 172.21.15.148:0/1034086792 learned_addr learned my addr 172.21.15.148:0/1034086792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb2ffd640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c004d40 0x7fbcb40c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb37fe640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb406bde0 0x7fbcb40d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 -- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb406bde0 msgr2=0x7fbcb40d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:53.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb406bde0 0x7fbcb40d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 -- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c004d40 msgr2=0x7fbcb40c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c004d40 0x7fbcb40c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 -- 172.21.15.148:0/1034086792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40c7a00 con 0x7fbcb415b460 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbcb27fc640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbca80119a0 tx=0x7fbca8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.997+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800faf0 con 0x7fbcb415b460 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.998+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800fc90 con 0x7fbcb415b460 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.998+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb40c7ce0 con 0x7fbcb415b460 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.998+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8014440 con 0x7fbcb415b460 2024-03-04T12:11:53.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.998+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb4151280 con 0x7fbcb415b460 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.999+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb4074f60 con 0x7fbcb415b460 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:52.999+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbca8012040 con 0x7fbcb415b460 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.000+0000 7fbc9b7fe640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 0x7fbca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.000+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbca808a4f0 con 0x7fbcb415b460 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.000+0000 7fbcb2ffd640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 0x7fbca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.000+0000 7fbcb2ffd640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 0x7fbca006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbca40013d0 tx=0x7fbca4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:53.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.004+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbca8019030 con 0x7fbcb415b460 2024-03-04T12:11:53.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.175+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fbcb40b5500 con 0x7fbcb415b460 2024-03-04T12:11:53.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.868+0000 7fbc9b7fe640 1 -- 172.21.15.148:0/1034086792 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fbca8058f90 con 0x7fbcb415b460 2024-03-04T12:11:53.869 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-03-04T12:11:53.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.871+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 msgr2=0x7fbca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:53.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.871+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 0x7fbca006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbca40013d0 tx=0x7fbca4006040 comp rx=0 tx=0).stop 2024-03-04T12:11:53.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.871+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 msgr2=0x7fbcb4150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:53.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.871+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbca80119a0 tx=0x7fbca8011e60 comp rx=0 tx=0).stop 2024-03-04T12:11:53.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 shutdown_connections 2024-03-04T12:11:53.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbca006b4d0 0x7fbca006d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcb406bde0 0x7fbcb40d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415b460 0x7fbcb4150d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 --2- 172.21.15.148:0/1034086792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9c004d40 0x7fbcb40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.872+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 >> 172.21.15.148:0/1034086792 conn(0x7fbcb4066dc0 msgr2=0x7fbcb40cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.873+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 shutdown_connections 2024-03-04T12:11:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:53.874+0000 7fbcb92d7640 1 -- 172.21.15.148:0/1034086792 wait complete. 2024-03-04T12:11:53.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-04T12:11:54.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.002+0000 7f86da436640 1 Processor -- start 2024-03-04T12:11:54.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.002+0000 7f86da436640 1 -- start start 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86da436640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 0x7f86d40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86da436640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d4162450 con 0x7f86d4161df0 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86da436640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d40cc2d0 con 0x7f86d40cf660 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86da436640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40cc920 con 0x7f86d415da20 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d89d3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f86d4161df0 0x7f86d406bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40412/0 (socket says 172.21.15.148:40412) 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d89d3640 1 -- 172.21.15.148:0/1887572950 learned_addr learned my addr 172.21.15.148:0/1887572950 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d3fff640 1 --2- 172.21.15.148:0/1887572950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 0x7f86d40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161136532 0 0) 0x7f86d4162450 con 0x7f86d4161df0 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86bc003520 con 0x7f86d4161df0 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349365431 0 0) 0x7f86d40cc2d0 con 0x7f86d40cf660 2024-03-04T12:11:54.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.003+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d4162450 con 0x7f86d40cf660 2024-03-04T12:11:54.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.004+0000 7f86d3fff640 1 -- 172.21.15.148:0/1887572950 >> v1:172.21.15.191:6789/0 conn(0x7f86d40cf660 legacy=0x7f86d41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.004+0000 7f86d3fff640 1 -- 172.21.15.148:0/1887572950 >> v1:172.21.15.148:6789/0 conn(0x7f86d4161df0 legacy=0x7f86d406bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.004+0000 7f86d3fff640 1 -- 172.21.15.148:0/1887572950 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d40cc5b0 con 0x7f86d415da20 2024-03-04T12:11:54.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.004+0000 7f86d3fff640 1 --2- 172.21.15.148:0/1887572950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 0x7f86d40ccdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097d0 tx=0x7f86c402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952961db4b0c007f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:54.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.004+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4035020 con 0x7f86d415da20 2024-03-04T12:11:54.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c4031890 con 0x7f86d415da20 2024-03-04T12:11:54.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86d2ffd640 1 -- 172.21.15.148:0/1887572950 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4031c40 con 0x7f86d415da20 2024-03-04T12:11:54.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 -- 172.21.15.148:0/1887572950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 msgr2=0x7f86d40ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 --2- 172.21.15.148:0/1887572950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 0x7f86d40ccdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097d0 tx=0x7f86c402b930 comp rx=0 tx=0).stop 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 -- 172.21.15.148:0/1887572950 shutdown_connections 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 --2- 172.21.15.148:0/1887572950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d415da20 0x7f86d40ccdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 -- 172.21.15.148:0/1887572950 >> 172.21.15.148:0/1887572950 conn(0x7f86d40b33a0 msgr2=0x7f86d40b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 -- 172.21.15.148:0/1887572950 shutdown_connections 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.005+0000 7f86da436640 1 -- 172.21.15.148:0/1887572950 wait complete. 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.006+0000 7f86da436640 1 Processor -- start 2024-03-04T12:11:54.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.006+0000 7f86da436640 1 -- start start 2024-03-04T12:11:54.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40cf660 0x7f86d41f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:54.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:54.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d4161df0 0x7f86d40cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:54.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d415de00 con 0x7f86d415da20 2024-03-04T12:11:54.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40cbf80 con 0x7f86d40cf660 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86da436640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d406c4f0 con 0x7f86d4161df0 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41354/0 (socket says 172.21.15.148:41354) 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86d37fe640 1 -- 172.21.15.148:0/1540388976 learned_addr learned my addr 172.21.15.148:0/1540388976 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86d89d3640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d4161df0 0x7f86d40cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.007+0000 7f86d3fff640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40cf660 0x7f86d41f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 -- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d4161df0 msgr2=0x7f86d40cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d4161df0 0x7f86d40cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 -- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40cf660 msgr2=0x7f86d41f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40cf660 0x7f86d41f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 -- 172.21.15.148:0/1540388976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d41f4a00 con 0x7f86d415da20 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d37fe640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c500 tx=0x7f86c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80052d0 con 0x7f86d415da20 2024-03-04T12:11:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c8005490 con 0x7f86d415da20 2024-03-04T12:11:54.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d40c6940 con 0x7f86d415da20 2024-03-04T12:11:54.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8012870 con 0x7f86d415da20 2024-03-04T12:11:54.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.008+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d40c6f00 con 0x7f86d415da20 2024-03-04T12:11:54.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.009+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d4074f60 con 0x7f86d415da20 2024-03-04T12:11:54.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.009+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f86c8012a10 con 0x7f86d415da20 2024-03-04T12:11:54.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.010+0000 7f86d0ff9640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:54.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.010+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f86c8089d40 con 0x7f86d415da20 2024-03-04T12:11:54.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.010+0000 7f86d3fff640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:54.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.010+0000 7f86d3fff640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 0x7f86c006d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86c4036000 tx=0x7f86c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:54.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.014+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86d41f4a00 con 0x7f86d415da20 2024-03-04T12:11:54.144 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:11:54.147 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:11:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.224+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86d4161490 con 0x7f86d415da20 2024-03-04T12:11:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.891+0000 7f86d0ff9640 1 -- 172.21.15.148:0/1540388976 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f86c80585e0 con 0x7f86d415da20 2024-03-04T12:11:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 2 and data pool 3 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.897+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 msgr2=0x7f86c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.897+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 0x7f86c006d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86c4036000 tx=0x7f86c4008030 comp rx=0 tx=0).stop 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.897+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 msgr2=0x7f86d4154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.897+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c500 tx=0x7f86c800c9c0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 shutdown_connections 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d4161df0 0x7f86d40cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d415da20 0x7f86d4154a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 --2- 172.21.15.148:0/1540388976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40cf660 0x7f86d41f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:54.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 >> 172.21.15.148:0/1540388976 conn(0x7f86d40b33a0 msgr2=0x7f86d40b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:54.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.898+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 shutdown_connections 2024-03-04T12:11:54.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:54.899+0000 7f86da436640 1 -- 172.21.15.148:0/1540388976 wait complete. 2024-03-04T12:11:54.922 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-04T12:11:54.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-04T12:11:55.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.072+0000 7f3cd01c4640 1 Processor -- start 2024-03-04T12:11:55.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.072+0000 7f3cd01c4640 1 -- start start 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cd01c4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3cc80b24d0 0x7f3cc80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cd01c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc80b4270 con 0x7f3cc815f1f0 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cd01c4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc80d07b0 con 0x7f3cc80b18a0 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cd01c4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80b1f10 con 0x7f3cc80b24d0 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3ccdf60640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3cc815f1f0 0x7f3cc80b1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40416/0 (socket says 172.21.15.148:40416) 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3ccdf60640 1 -- 172.21.15.148:0/566292751 learned_addr learned my addr 172.21.15.148:0/566292751 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540064202 0 0) 0x7f3cc4001110 con 0x7f3cc815f1f0 2024-03-04T12:11:55.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cce761640 1 --2- 172.21.15.148:0/566292751 >> v2:172.21.15.191:3300/0 conn(0x7f3cc80b24d0 0x7f3cc80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb4003520 con 0x7f3cc815f1f0 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962446632 0 0) 0x7f3cb4003520 con 0x7f3cc815f1f0 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 >> v2:172.21.15.191:3300/0 conn(0x7f3cc80b24d0 msgr2=0x7f3cc80cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 --2- 172.21.15.148:0/566292751 >> v2:172.21.15.191:3300/0 conn(0x7f3cc80b24d0 0x7f3cc80cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 >> v1:172.21.15.191:6789/0 conn(0x7f3cc80b18a0 legacy=0x7f3cc80cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80b21f0 con 0x7f3cc815f1f0 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f3cc4001840 con 0x7f3cc815f1f0 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 >> v1:172.21.15.148:6789/0 conn(0x7f3cc815f1f0 legacy=0x7f3cc80b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cb40039b0 0x7f3cb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cb4004800 0x7f3cb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.073+0000 7f3cccf5e640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b1f10 con 0x7f3cb40051c0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb400c060 con 0x7f3cb4004800 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb400c260 con 0x7f3cb40039b0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3ccd75f640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cb4004800 0x7f3cb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3ccdf60640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cb40039b0 0x7f3cb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 -- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cb40039b0 msgr2=0x7f3cb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cb40039b0 0x7f3cb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 -- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cb4004800 msgr2=0x7f3cb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cb4004800 0x7f3cb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 -- 172.21.15.148:0/566292751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4003520 con 0x7f3cb40051c0 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.074+0000 7f3cce761640 1 -- 172.21.15.148:0/566292751 reap_dead start 2024-03-04T12:11:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cce761640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cb400b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009fc0 tx=0x7f3cb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee362dd9952353da server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0009c10 con 0x7f3cb40051c0 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc002cca0 con 0x7f3cb40051c0 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cccf5e640 1 -- 172.21.15.148:0/566292751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003bd0 con 0x7f3cb40051c0 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 -- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 msgr2=0x7f3cb400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cb400b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009fc0 tx=0x7f3cb4005830 comp rx=0 tx=0).stop 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 -- 172.21.15.148:0/566292751 shutdown_connections 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/566292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cb400b9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 -- 172.21.15.148:0/566292751 >> 172.21.15.148:0/566292751 conn(0x7f3cc80b3320 msgr2=0x7f3cc80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 -- 172.21.15.148:0/566292751 shutdown_connections 2024-03-04T12:11:55.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.075+0000 7f3cd01c4640 1 -- 172.21.15.148:0/566292751 wait complete. 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.076+0000 7f3cd01c4640 1 Processor -- start 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.076+0000 7f3cd01c4640 1 -- start start 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cc80c7150 0x7f3cc814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cc80c75d0 0x7f3cc814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b28e0 con 0x7f3cb40051c0 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80d0290 con 0x7f3cc80c75d0 2024-03-04T12:11:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cd01c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b1190 con 0x7f3cc80c7150 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41374/0 (socket says 172.21.15.148:41374) 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 -- 172.21.15.148:0/1665628757 learned_addr learned my addr 172.21.15.148:0/1665628757 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3cce761640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cc80c75d0 0x7f3cc814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccd75f640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cc80c7150 0x7f3cc814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 -- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cc80c7150 msgr2=0x7f3cc814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cc80c7150 0x7f3cc814ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 -- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cc80c75d0 msgr2=0x7f3cc814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cc80c75d0 0x7f3cc814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.077+0000 7f3ccdf60640 1 -- 172.21.15.148:0/1665628757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80b4580 con 0x7f3cb40051c0 2024-03-04T12:11:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3ccdf60640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4007fd0 tx=0x7f3cc40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:55.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4007740 con 0x7f3cb40051c0 2024-03-04T12:11:55.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4018070 con 0x7f3cb40051c0 2024-03-04T12:11:55.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc80b4860 con 0x7f3cb40051c0 2024-03-04T12:11:55.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400fb90 con 0x7f3cb40051c0 2024-03-04T12:11:55.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.078+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc80b5340 con 0x7f3cb40051c0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.079+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc8074f60 con 0x7f3cb40051c0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.079+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3cc400fd30 con 0x7f3cb40051c0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.080+0000 7f3cbe7fc640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 0x7f3cac06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.080+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cc408a1d0 con 0x7f3cb40051c0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.080+0000 7f3ccd75f640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 0x7f3cac06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.080+0000 7f3ccd75f640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 0x7f3cac06db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8000c00 tx=0x7f3cb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:55.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.084+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cc80b4270 con 0x7f3cb40051c0 2024-03-04T12:11:55.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.265+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3cc809a900 con 0x7f3cb40051c0 2024-03-04T12:11:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.892+0000 7f3cbe7fc640 1 -- 172.21.15.148:0/1665628757 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3cc40590e0 con 0x7f3cb40051c0 2024-03-04T12:11:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 msgr2=0x7f3cac06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 0x7f3cac06db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8000c00 tx=0x7f3cb8009040 comp rx=0 tx=0).stop 2024-03-04T12:11:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 msgr2=0x7f3cc80c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4007fd0 tx=0x7f3cc40074f0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 shutdown_connections 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3cac06b6a0 0x7f3cac06db60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cc80c75d0 0x7f3cc814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cc80c7150 0x7f3cc814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 --2- 172.21.15.148:0/1665628757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb40051c0 0x7f3cc80c54b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 >> 172.21.15.148:0/1665628757 conn(0x7f3cc80b3320 msgr2=0x7f3cc806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 shutdown_connections 2024-03-04T12:11:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:55.894+0000 7f3cd01c4640 1 -- 172.21.15.148:0/1665628757 wait complete. 2024-03-04T12:11:55.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-04T12:11:56.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.017+0000 7fbcd857e640 1 Processor -- start 2024-03-04T12:11:56.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.017+0000 7fbcd857e640 1 -- start start 2024-03-04T12:11:56.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.018+0000 7fbcd857e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd01450e0 0x7fbcd00cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.018+0000 7fbcd857e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbcd015c9b0 0x7fbcd00cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.018+0000 7fbcd857e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd00cebd0 con 0x7fbcd00cc340 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.018+0000 7fbcd857e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd00cf7e0 con 0x7fbcd015c9b0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.018+0000 7fbcd857e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00cc780 con 0x7fbcd01450e0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbcd015c9b0 0x7fbcd00cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbcd015c9b0 0x7fbcd00cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41378/0 (socket says 172.21.15.148:41378) 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 -- 172.21.15.148:0/1356353052 learned_addr learned my addr 172.21.15.148:0/1356353052 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd631a640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd01450e0 0x7fbcd00cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 -- 172.21.15.148:0/1356353052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd01450e0 msgr2=0x7fbcd00cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd01450e0 0x7fbcd00cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 -- 172.21.15.148:0/1356353052 >> v1:172.21.15.191:6789/0 conn(0x7fbcd00cc340 legacy=0x7fbcd00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.019+0000 7fbcd5b19640 1 -- 172.21.15.148:0/1356353052 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00ceeb0 con 0x7fbcd015c9b0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd015c9b0 0x7fbcd00cbe00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009870 tx=0x7fbcc002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d4b5bc807e5f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd5318640 1 -- 172.21.15.148:0/1356353052 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0035020 con 0x7fbcd015c9b0 2024-03-04T12:11:56.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd5318640 1 -- 172.21.15.148:0/1356353052 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0031640 con 0x7fbcd015c9b0 2024-03-04T12:11:56.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd5318640 1 -- 172.21.15.148:0/1356353052 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc002d040 con 0x7fbcd015c9b0 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 -- 172.21.15.148:0/1356353052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd015c9b0 msgr2=0x7fbcd00cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd015c9b0 0x7fbcd00cbe00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009870 tx=0x7fbcc002b8f0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 -- 172.21.15.148:0/1356353052 shutdown_connections 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd015c9b0 0x7fbcd00cbe00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 --2- 172.21.15.148:0/1356353052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd01450e0 0x7fbcd00cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 -- 172.21.15.148:0/1356353052 >> 172.21.15.148:0/1356353052 conn(0x7fbcd006acd0 msgr2=0x7fbcd006b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 -- 172.21.15.148:0/1356353052 shutdown_connections 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.020+0000 7fbcd857e640 1 -- 172.21.15.148:0/1356353052 wait complete. 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 Processor -- start 2024-03-04T12:11:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 -- start start 2024-03-04T12:11:56.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 0x7fbcd01e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 0x7fbcd01e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01454c0 con 0x7fbcd00cc340 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00cf470 con 0x7fbcd01450e0 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd857e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd015ce30 con 0x7fbcd015c9b0 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd631a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd631a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41392/0 (socket says 172.21.15.148:41392) 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.022+0000 7fbcd631a640 1 -- 172.21.15.148:0/2814542404 learned_addr learned my addr 172.21.15.148:0/2814542404 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 0x7fbcd01e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd6b1b640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 0x7fbcd01e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 -- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 msgr2=0x7fbcd01e9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 0x7fbcd01e9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 -- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 msgr2=0x7fbcd01e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 0x7fbcd01e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 -- 172.21.15.148:0/2814542404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0140970 con 0x7fbcd00cc340 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd6b1b640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 0x7fbcd01e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 0x7fbcd01e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd631a640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c820 tx=0x7fbccc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc015070 con 0x7fbcd00cc340 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc004240 con 0x7fbcd00cc340 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00ce10 con 0x7fbcd00cc340 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0140c50 con 0x7fbcd00cc340 2024-03-04T12:11:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.023+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd0141210 con 0x7fbcd00cc340 2024-03-04T12:11:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.024+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0074f60 con 0x7fbcd00cc340 2024-03-04T12:11:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.025+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbccc007830 con 0x7fbcd00cc340 2024-03-04T12:11:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.025+0000 7fbcc6ffd640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 0x7fbcb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.025+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fbccc08a2a0 con 0x7fbcd00cc340 2024-03-04T12:11:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.025+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 0x7fbcb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.025+0000 7fbcd5b19640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 0x7fbcb406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0036000 tx=0x7fbcc0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:56.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.029+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbccc058f60 con 0x7fbcd00cc340 2024-03-04T12:11:56.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:56.205+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbcd00cc120 con 0x7fbcd00cc340 2024-03-04T12:11:57.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.912+0000 7fbcc6ffd640 1 -- 172.21.15.148:0/2814542404 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbccc058860 con 0x7fbcd00cc340 2024-03-04T12:11:57.913 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' created 2024-03-04T12:11:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.916+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 msgr2=0x7fbcb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.916+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 0x7fbcb406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0036000 tx=0x7fbcc0008030 comp rx=0 tx=0).stop 2024-03-04T12:11:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.916+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 msgr2=0x7fbcd015b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.916+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c820 tx=0x7fbccc00cce0 comp rx=0 tx=0).stop 2024-03-04T12:11:57.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 shutdown_connections 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbcb406b4d0 0x7fbcb406d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbcd015c9b0 0x7fbcd01e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcd01450e0 0x7fbcd01e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 --2- 172.21.15.148:0/2814542404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd00cc340 0x7fbcd015b3d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.918+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 >> 172.21.15.148:0/2814542404 conn(0x7fbcd006acd0 msgr2=0x7fbcd00b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.919+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 shutdown_connections 2024-03-04T12:11:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:57.919+0000 7fbcd857e640 1 -- 172.21.15.148:0/2814542404 wait complete. 2024-03-04T12:11:57.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-04T12:11:58.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.044+0000 7f6e4f5a9640 1 Processor -- start 2024-03-04T12:11:58.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.044+0000 7f6e4f5a9640 1 -- start start 2024-03-04T12:11:58.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4f5a9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 0x7f6e4809a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4f5a9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e4815bf10 0x7f6e480cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4f5a9640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480cd360 con 0x7f6e480d0200 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4f5a9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4809aea0 con 0x7f6e4815bf10 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4f5a9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815ca80 con 0x7f6e480cfe20 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e4815bf10 0x7f6e480cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e4815bf10 0x7f6e480cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41400/0 (socket says 172.21.15.148:41400) 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 -- 172.21.15.148:0/121612746 learned_addr learned my addr 172.21.15.148:0/121612746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4d345640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 0x7f6e4809a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 -- 172.21.15.148:0/121612746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 msgr2=0x7f6e4809a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 0x7f6e4809a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 -- 172.21.15.148:0/121612746 >> v1:172.21.15.191:6789/0 conn(0x7f6e480d0200 legacy=0x7f6e4815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4cb44640 1 -- 172.21.15.148:0/121612746 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480cfaa0 con 0x7f6e4815bf10 2024-03-04T12:11:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.045+0000 7f6e4d345640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 0x7f6e4809a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.046+0000 7f6e4cb44640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e480cf410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e08995c94f95988 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.046+0000 7f6e3ffff640 1 -- 172.21.15.148:0/121612746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380317c0 con 0x7f6e4815bf10 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.046+0000 7f6e3ffff640 1 -- 172.21.15.148:0/121612746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38037070 con 0x7f6e4815bf10 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.046+0000 7f6e3ffff640 1 -- 172.21.15.148:0/121612746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38031b80 con 0x7f6e4815bf10 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/121612746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 msgr2=0x7f6e480cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e480cf410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3802b860 comp rx=0 tx=0).stop 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/121612746 shutdown_connections 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e480cf410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/121612746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480cfe20 0x7f6e4809a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/121612746 >> 172.21.15.148:0/121612746 conn(0x7f6e480d1840 msgr2=0x7f6e480d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:58.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/121612746 shutdown_connections 2024-03-04T12:11:58.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.047+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/121612746 wait complete. 2024-03-04T12:11:58.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.048+0000 7f6e4f5a9640 1 Processor -- start 2024-03-04T12:11:58.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.048+0000 7f6e4f5a9640 1 -- start start 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e480cfe20 0x7f6e48154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480d0200 0x7f6e48153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480d06b0 con 0x7f6e4815bf10 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815c710 con 0x7f6e480cfe20 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4f5a9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815cf10 con 0x7f6e480d0200 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41410/0 (socket says 172.21.15.148:41410) 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 -- 172.21.15.148:0/3367897580 learned_addr learned my addr 172.21.15.148:0/3367897580 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4cb44640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480d0200 0x7f6e48153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4d345640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e480cfe20 0x7f6e48154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 -- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480d0200 msgr2=0x7f6e48153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480d0200 0x7f6e48153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 -- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e480cfe20 msgr2=0x7f6e48154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e480cfe20 0x7f6e48154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 -- 172.21.15.148:0/3367897580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480b46c0 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e4db46640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b410 tx=0x7f6e4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.049+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40015030 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.050+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40004020 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.050+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480b4980 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.050+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40004410 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.050+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e480b4f40 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.051+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48074f60 con 0x7f6e4815bf10 2024-03-04T12:11:58.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.051+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6e40018070 con 0x7f6e4815bf10 2024-03-04T12:11:58.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.051+0000 7f6e3dffb640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:58.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.051+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6e40089fa0 con 0x7f6e4815bf10 2024-03-04T12:11:58.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.052+0000 7f6e4d345640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:58.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.052+0000 7f6e4d345640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007510 tx=0x7f6e440074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:58.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.056+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e40058b90 con 0x7f6e4815bf10 2024-03-04T12:11:58.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.228+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f6e480cdc50 con 0x7f6e4815bf10 2024-03-04T12:11:58.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.915+0000 7f6e3dffb640 1 -- 172.21.15.148:0/3367897580 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e40058490 con 0x7f6e4815bf10 2024-03-04T12:11:58.917 INFO:teuthology.orchestra.run.smithi148.stderr:set pool 4 allow_ec_overwrites to true 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 msgr2=0x7f6e2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007510 tx=0x7f6e440074a0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 msgr2=0x7f6e48155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b410 tx=0x7f6e4000b8d0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 shutdown_connections 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007510 tx=0x7f6e440074a0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4815bf10 0x7f6e48155a30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e480d0200 0x7f6e48153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 --2- 172.21.15.148:0/3367897580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e480cfe20 0x7f6e48154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:58.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.919+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 >> 172.21.15.148:0/3367897580 conn(0x7f6e480d1840 msgr2=0x7f6e4806f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:58.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.920+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 shutdown_connections 2024-03-04T12:11:58.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:58.920+0000 7f6e4f5a9640 1 -- 172.21.15.148:0/3367897580 wait complete. 2024-03-04T12:11:58.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-04T12:11:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.052+0000 7f29a00bf640 1 Processor -- start 2024-03-04T12:11:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.053+0000 7f29a00bf640 1 -- start start 2024-03-04T12:11:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.053+0000 7f29a00bf640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f299815a810 0x7f29980b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.053+0000 7f29a00bf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f299806d3c0 0x7f29980b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f29a00bf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299806d8a0 con 0x7f29980cbfb0 2024-03-04T12:11:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f29a00bf640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299806db80 con 0x7f299806d3c0 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f29a00bf640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299806de60 con 0x7f299815a810 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299d65a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f299806d3c0 0x7f29980b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299e65c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f29980cbfb0 0x7f299815abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40420/0 (socket says 172.21.15.148:40420) 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299e65c640 1 -- 172.21.15.148:0/471906751 learned_addr learned my addr 172.21.15.148:0/471906751 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299de5b640 1 --2- 172.21.15.148:0/471906751 >> v2:172.21.15.191:3300/0 conn(0x7f299815a810 0x7f29980b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299d65a640 1 -- 172.21.15.148:0/471906751 >> v2:172.21.15.191:3300/0 conn(0x7f299815a810 msgr2=0x7f29980b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299d65a640 1 --2- 172.21.15.148:0/471906751 >> v2:172.21.15.191:3300/0 conn(0x7f299815a810 0x7f29980b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299d65a640 1 -- 172.21.15.148:0/471906751 >> v1:172.21.15.148:6789/0 conn(0x7f29980cbfb0 legacy=0x7f299815abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299d65a640 1 -- 172.21.15.148:0/471906751 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299806e140 con 0x7f299806d3c0 2024-03-04T12:11:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.054+0000 7f299ce59640 1 -- 172.21.15.148:0/471906751 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838361848 0 0) 0x7f299806d8a0 con 0x7f29980cbfb0 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f299d65a640 1 --2- 172.21.15.148:0/471906751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f29980b46d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f299806e560 tx=0x7f298802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3059abb7e0c697 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f299ce59640 1 -- 172.21.15.148:0/471906751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988035020 con 0x7f299806d3c0 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f299ce59640 1 -- 172.21.15.148:0/471906751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2988031830 con 0x7f299806d3c0 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f299ce59640 1 -- 172.21.15.148:0/471906751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988031c00 con 0x7f299806d3c0 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 -- 172.21.15.148:0/471906751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 msgr2=0x7f29980b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 --2- 172.21.15.148:0/471906751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f29980b46d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f299806e560 tx=0x7f298802b8d0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 -- 172.21.15.148:0/471906751 shutdown_connections 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 --2- 172.21.15.148:0/471906751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f29980b46d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 --2- 172.21.15.148:0/471906751 >> v2:172.21.15.191:3300/0 conn(0x7f299815a810 0x7f29980b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 -- 172.21.15.148:0/471906751 >> 172.21.15.148:0/471906751 conn(0x7f299806acd0 msgr2=0x7f299806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 -- 172.21.15.148:0/471906751 shutdown_connections 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.055+0000 7f29a00bf640 1 -- 172.21.15.148:0/471906751 wait complete. 2024-03-04T12:11:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 Processor -- start 2024-03-04T12:11:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 -- start start 2024-03-04T12:11:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 0x7f29980cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 0x7f29980ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299815e3c0 con 0x7f299806d3c0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b3c50 con 0x7f299815a810 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f29a00bf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299815e540 con 0x7f29980cbfb0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f299de5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f299de5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41430/0 (socket says 172.21.15.148:41430) 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.057+0000 7f299de5b640 1 -- 172.21.15.148:0/3767968344 learned_addr learned my addr 172.21.15.148:0/3767968344 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299e65c640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 0x7f29980ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 -- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 msgr2=0x7f29980cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299d65a640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 0x7f29980cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 0x7f29980cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 -- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 msgr2=0x7f29980ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 0x7f29980ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 -- 172.21.15.148:0/3767968344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299814cf80 con 0x7f299806d3c0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299e65c640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 0x7f29980ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299d65a640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 0x7f29980cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f299de5b640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2990007e40 tx=0x7f299000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990014620 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990019070 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299814d260 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990014a00 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.058+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299814d820 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.059+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2998074f60 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.059+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2990014ba0 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.060+0000 7f298e7fc640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 0x7f297c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.060+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f299008ad30 con 0x7f299806d3c0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.060+0000 7f299d65a640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 0x7f297c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:11:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.060+0000 7f299d65a640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 0x7f297c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2988036000 tx=0x7f2988008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:11:59.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.064+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2990059920 con 0x7f299806d3c0 2024-03-04T12:11:59.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.271+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f299814b240 con 0x7f299806d3c0 2024-03-04T12:11:59.394 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-04T12:11:59.392+0000 7f453a2a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T12:11:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.922+0000 7f298e7fc640 1 -- 172.21.15.148:0/3767968344 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2990059220 con 0x7f299806d3c0 2024-03-04T12:11:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:added data pool 4 to fsmap 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 msgr2=0x7f297c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 0x7f297c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2988036000 tx=0x7f2988008030 comp rx=0 tx=0).stop 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 msgr2=0x7f2998147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2990007e40 tx=0x7f299000e4b0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 shutdown_connections 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f297c06b4d0 0x7f297c06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f299815a810 0x7f29980ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29980cbfb0 0x7f29980cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 --2- 172.21.15.148:0/3767968344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f299806d3c0 0x7f2998147520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:11:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 >> 172.21.15.148:0/3767968344 conn(0x7f299806acd0 msgr2=0x7f29980cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:11:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 shutdown_connections 2024-03-04T12:11:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:11:59.924+0000 7f29a00bf640 1 -- 172.21.15.148:0/3767968344 wait complete. 2024-03-04T12:11:59.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.039+0000 7fbb69859640 1 Processor -- start 2024-03-04T12:12:00.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.039+0000 7fbb69859640 1 -- start start 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb69859640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb6415de00 0x7fbb640b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb69859640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6415e1e0 con 0x7fbb640b5470 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb69859640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d1c10 con 0x7fbb640b9380 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb69859640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640d2820 con 0x7fbb6415de00 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb637fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb640b5470 0x7fbb640d2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40424/0 (socket says 172.21.15.148:40424) 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb637fe640 1 -- 172.21.15.148:0/2970813063 learned_addr learned my addr 172.21.15.148:0/2970813063 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb62ffd640 1 --2- 172.21.15.148:0/2970813063 >> v2:172.21.15.191:3300/0 conn(0x7fbb6415de00 0x7fbb640b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253720243 0 0) 0x7fbb6415e1e0 con 0x7fbb640b5470 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.040+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb50003520 con 0x7fbb640b5470 2024-03-04T12:12:00.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2889289362 0 0) 0x7fbb50003520 con 0x7fbb640b5470 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 >> v2:172.21.15.191:3300/0 conn(0x7fbb6415de00 msgr2=0x7fbb640b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 --2- 172.21.15.148:0/2970813063 >> v2:172.21.15.191:3300/0 conn(0x7fbb6415de00 0x7fbb640b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 >> v1:172.21.15.191:6789/0 conn(0x7fbb640b9380 legacy=0x7fbb640b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640d1ef0 con 0x7fbb640b5470 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fbb5c0019f0 con 0x7fbb640b5470 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 >> v1:172.21.15.148:6789/0 conn(0x7fbb640b5470 legacy=0x7fbb640d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb500039b0 0x7fbb50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 0x7fbb50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb500051c0 0x7fbb5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640d2820 con 0x7fbb50004800 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5000c060 con 0x7fbb500051c0 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5000c260 con 0x7fbb500039b0 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 0x7fbb50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb637fe640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb500051c0 0x7fbb5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb62ffd640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb500039b0 0x7fbb50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb500039b0 msgr2=0x7fbb50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb500039b0 0x7fbb50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb500051c0 msgr2=0x7fbb5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb500051c0 0x7fbb5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/2970813063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50003520 con 0x7fbb50004800 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.041+0000 7fbb637fe640 1 -- 172.21.15.148:0/2970813063 reap_dead start 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 0x7fbb50004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c001750 tx=0x7fbb50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0582eaa9983588 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c001370 con 0x7fbb50004800 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c0052d0 con 0x7fbb50004800 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb627fc640 1 -- 172.21.15.148:0/2970813063 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c003830 con 0x7fbb50004800 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb69859640 1 -- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 msgr2=0x7fbb50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.042+0000 7fbb69859640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 0x7fbb50004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c001750 tx=0x7fbb50005830 comp rx=0 tx=0).stop 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.043+0000 7fbb69859640 1 -- 172.21.15.148:0/2970813063 shutdown_connections 2024-03-04T12:12:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.043+0000 7fbb69859640 1 --2- 172.21.15.148:0/2970813063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50004800 0x7fbb50004c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.043+0000 7fbb69859640 1 -- 172.21.15.148:0/2970813063 >> 172.21.15.148:0/2970813063 conn(0x7fbb640d0470 msgr2=0x7fbb640d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.043+0000 7fbb69859640 1 -- 172.21.15.148:0/2970813063 shutdown_connections 2024-03-04T12:12:00.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.043+0000 7fbb69859640 1 -- 172.21.15.148:0/2970813063 wait complete. 2024-03-04T12:12:00.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.044+0000 7fbb69859640 1 Processor -- start 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.044+0000 7fbb69859640 1 -- start start 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50004800 0x7fbb64152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641595f0 0x7fbb64159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b97e0 con 0x7fbb64154040 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640d24b0 con 0x7fbb641595f0 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b2e60 con 0x7fbb50004800 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41452/0 (socket says 172.21.15.148:41452) 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/3397437308 learned_addr learned my addr 172.21.15.148:0/3397437308 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb62ffd640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50004800 0x7fbb64152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb637fe640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641595f0 0x7fbb64159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50004800 msgr2=0x7fbb64152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50004800 0x7fbb64152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641595f0 msgr2=0x7fbb64159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641595f0 0x7fbb64159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 -- 172.21.15.148:0/3397437308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64156110 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb5a7fc640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c001870 tx=0x7fbb4c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c041070 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c005480 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c007a30 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb641563f0 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.045+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb641569b0 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.046+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64074f60 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.047+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb4c044070 con 0x7fbb64154040 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.047+0000 7fbb60ff9640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.047+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbb4c0abeb0 con 0x7fbb64154040 2024-03-04T12:12:00.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.047+0000 7fbb62ffd640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.048+0000 7fbb62ffd640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 0x7fbb4406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009c40 tx=0x7fbb54009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.051+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb4c07a3a0 con 0x7fbb64154040 2024-03-04T12:12:00.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.258+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb6415eed0 con 0x7fbb64154040 2024-03-04T12:12:00.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.259+0000 7fbb60ff9640 1 -- 172.21.15.148:0/3397437308 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fbb4c07e440 con 0x7fbb64154040 2024-03-04T12:12:00.260 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:00.260 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:11:59.272589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T12:12:00.261 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 6 2024-03-04T12:12:00.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 msgr2=0x7fbb4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 0x7fbb4406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009c40 tx=0x7fbb54009190 comp rx=0 tx=0).stop 2024-03-04T12:12:00.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 msgr2=0x7fbb641544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c001870 tx=0x7fbb4c001120 comp rx=0 tx=0).stop 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 shutdown_connections 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb4406b4d0 0x7fbb4406d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641595f0 0x7fbb64159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb64154040 0x7fbb641544a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 --2- 172.21.15.148:0/3397437308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50004800 0x7fbb64152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 >> 172.21.15.148:0/3397437308 conn(0x7fbb640d0470 msgr2=0x7fbb640d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 shutdown_connections 2024-03-04T12:12:00.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.261+0000 7fbb69859640 1 -- 172.21.15.148:0/3397437308 wait complete. 2024-03-04T12:12:00.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:00.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.373+0000 7fc7d4359640 1 Processor -- start 2024-03-04T12:12:00.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.373+0000 7fc7d4359640 1 -- start start 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 0x7fc7cc0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b70d0 0x7fc7cc0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc0d2820 con 0x7fc7cc0b9380 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc15e1e0 con 0x7fc7cc15de00 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d4359640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0d1c10 con 0x7fc7cc0b70d0 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41462/0 (socket says 172.21.15.148:41462) 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 -- 172.21.15.148:0/3435560816 learned_addr learned my addr 172.21.15.148:0/3435560816 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d28f6640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b70d0 0x7fc7cc0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d20f5640 1 --2- 172.21.15.148:0/3435560816 >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 0x7fc7cc0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 -- 172.21.15.148:0/3435560816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b70d0 msgr2=0x7fc7cc0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b70d0 0x7fc7cc0b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 -- 172.21.15.148:0/3435560816 >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 msgr2=0x7fc7cc0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 --2- 172.21.15.148:0/3435560816 >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 0x7fc7cc0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d18f4640 1 -- 172.21.15.148:0/3435560816 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0d1ef0 con 0x7fc7cc0b9380 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.374+0000 7fc7d20f5640 1 --2- 172.21.15.148:0/3435560816 >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 0x7fc7cc0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d18f4640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ce240 tx=0x7fc7c802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd2ee61650a01e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d10f3640 1 -- 172.21.15.148:0/3435560816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8035020 con 0x7fc7cc0b9380 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d10f3640 1 -- 172.21.15.148:0/3435560816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c802d040 con 0x7fc7cc0b9380 2024-03-04T12:12:00.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d10f3640 1 -- 172.21.15.148:0/3435560816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8031a80 con 0x7fc7cc0b9380 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 -- 172.21.15.148:0/3435560816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc0b9380 msgr2=0x7fc7cc0b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ce240 tx=0x7fc7c802b900 comp rx=0 tx=0).stop 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 -- 172.21.15.148:0/3435560816 shutdown_connections 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b70d0 0x7fc7cc0b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 --2- 172.21.15.148:0/3435560816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc0b9380 0x7fc7cc0b6b90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 --2- 172.21.15.148:0/3435560816 >> v2:172.21.15.191:3300/0 conn(0x7fc7cc15de00 0x7fc7cc0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 -- 172.21.15.148:0/3435560816 >> 172.21.15.148:0/3435560816 conn(0x7fc7cc0d0470 msgr2=0x7fc7cc0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 -- 172.21.15.148:0/3435560816 shutdown_connections 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.375+0000 7fc7d4359640 1 -- 172.21.15.148:0/3435560816 wait complete. 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 Processor -- start 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 -- start start 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0b70d0 0x7fc7cc159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b9380 0x7fc7cc0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0b97e0 con 0x7fc7cc15de00 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0d24b0 con 0x7fc7cc0b70d0 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d4359640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0b59b0 con 0x7fc7cc0b9380 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d28f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d28f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41476/0 (socket says 172.21.15.148:41476) 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d28f6640 1 -- 172.21.15.148:0/4257714307 learned_addr learned my addr 172.21.15.148:0/4257714307 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d20f5640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0b70d0 0x7fc7cc159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.377+0000 7fc7d18f4640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b9380 0x7fc7cc0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 -- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b9380 msgr2=0x7fc7cc0b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b9380 0x7fc7cc0b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 -- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0b70d0 msgr2=0x7fc7cc159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0b70d0 0x7fc7cc159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 -- 172.21.15.148:0/4257714307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc1f32f0 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d28f6640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007e90 tx=0x7fc7c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4014420 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4019070 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc0cbd00 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4014820 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.378+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc0cc2c0 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.379+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc074f60 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.379+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc7c40149c0 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.380+0000 7fc7c2ffd640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.380+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7c408abf0 con 0x7fc7cc15de00 2024-03-04T12:12:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.380+0000 7fc7d20f5640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.380+0000 7fc7d20f5640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 0x7fc7b006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00aac0 tx=0x7fc7bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.384+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7c4059170 con 0x7fc7cc15de00 2024-03-04T12:12:00.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.558+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7cc0b8c30 con 0x7fc7cc15de00 2024-03-04T12:12:00.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.559+0000 7fc7c2ffd640 1 -- 172.21.15.148:0/4257714307 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7fc7c405d210 con 0x7fc7cc15de00 2024-03-04T12:12:00.561 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:00.561 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 msgr2=0x7fc7b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 0x7fc7b006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00aac0 tx=0x7fc7bc008040 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 msgr2=0x7fc7cc0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007e90 tx=0x7fc7c400e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 shutdown_connections 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7cc15de00 0x7fc7cc0b5170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0b9380 0x7fc7cc0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 --2- 172.21.15.148:0/4257714307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0b70d0 0x7fc7cc159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.561+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 >> 172.21.15.148:0/4257714307 conn(0x7fc7cc0d0470 msgr2=0x7fc7cc15d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.562+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 shutdown_connections 2024-03-04T12:12:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.562+0000 7fc7d4359640 1 -- 172.21.15.148:0/4257714307 wait complete. 2024-03-04T12:12:00.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.673+0000 7f8850a59640 1 Processor -- start 2024-03-04T12:12:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.673+0000 7f8850a59640 1 -- start start 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8850a59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f884c0cf650 0x7f884c0cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8850a59640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 0x7f884c071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8850a59640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c15f020 con 0x7f884c154a80 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8850a59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0cd010 con 0x7f884c0cf650 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8850a59640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0cdaa0 con 0x7f884c070de0 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f884c0cf650 0x7f884c0cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f884c0cf650 0x7f884c0cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41486/0 (socket says 172.21.15.148:41486) 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 -- 172.21.15.148:0/1883944367 learned_addr learned my addr 172.21.15.148:0/1883944367 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f884ad9d640 1 --2- 172.21.15.148:0/1883944367 >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 0x7f884c071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 -- 172.21.15.148:0/1883944367 >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 msgr2=0x7f884c071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 --2- 172.21.15.148:0/1883944367 >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 0x7f884c071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 -- 172.21.15.148:0/1883944367 >> v1:172.21.15.191:6789/0 conn(0x7f884c154a80 legacy=0x7f884c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f8843fff640 1 -- 172.21.15.148:0/1883944367 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0cdd80 con 0x7f884c0cf650 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.674+0000 7f884ad9d640 1 --2- 172.21.15.148:0/1883944367 >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 0x7f884c071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.675+0000 7f8843fff640 1 --2- 172.21.15.148:0/1883944367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c0cf650 0x7f884c0cfae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884c158690 tx=0x7f883402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebdeb11d231f374 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.675+0000 7f8849d9b640 1 -- 172.21.15.148:0/1883944367 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834035020 con 0x7f884c0cf650 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.675+0000 7f8849d9b640 1 -- 172.21.15.148:0/1883944367 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834031870 con 0x7f884c0cf650 2024-03-04T12:12:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.675+0000 7f8849d9b640 1 -- 172.21.15.148:0/1883944367 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834031c40 con 0x7f884c0cf650 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 -- 172.21.15.148:0/1883944367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c0cf650 msgr2=0x7f884c0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 --2- 172.21.15.148:0/1883944367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c0cf650 0x7f884c0cfae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884c158690 tx=0x7f883402b910 comp rx=0 tx=0).stop 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 -- 172.21.15.148:0/1883944367 shutdown_connections 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 --2- 172.21.15.148:0/1883944367 >> v2:172.21.15.191:3300/0 conn(0x7f884c070de0 0x7f884c071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 --2- 172.21.15.148:0/1883944367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c0cf650 0x7f884c0cfae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 -- 172.21.15.148:0/1883944367 >> 172.21.15.148:0/1883944367 conn(0x7f884c06f5f0 msgr2=0x7f884c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 -- 172.21.15.148:0/1883944367 shutdown_connections 2024-03-04T12:12:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.676+0000 7f8850a59640 1 -- 172.21.15.148:0/1883944367 wait complete. 2024-03-04T12:12:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.677+0000 7f8850a59640 1 Processor -- start 2024-03-04T12:12:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.677+0000 7f8850a59640 1 -- start start 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f884c070de0 0x7f884c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f884c0cf650 0x7f884c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0ce620 con 0x7f884c154a80 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0ce340 con 0x7f884c0cf650 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c154e60 con 0x7f884c070de0 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41502/0 (socket says 172.21.15.148:41502) 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 -- 172.21.15.148:0/1913238265 learned_addr learned my addr 172.21.15.148:0/1913238265 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884a59c640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f884c070de0 0x7f884c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8843fff640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f884c0cf650 0x7f884c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 -- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f884c070de0 msgr2=0x7f884c1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f884c070de0 0x7f884c1538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 -- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f884c0cf650 msgr2=0x7f884c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f884c0cf650 0x7f884c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 -- 172.21.15.148:0/1913238265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c155870 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f884ad9d640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8844007f20 tx=0x7f884400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884401a070 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844003b90 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004030 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.678+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c153e20 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.679+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c1e7580 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.680+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c074f60 con 0x7f884c154a80 2024-03-04T12:12:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.680+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f884401d070 con 0x7f884c154a80 2024-03-04T12:12:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.680+0000 7f8842ffd640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 0x7f882c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.680+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f884408a830 con 0x7f884c154a80 2024-03-04T12:12:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.680+0000 7f884a59c640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 0x7f882c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.681+0000 7f884a59c640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 0x7f882c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.685+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8844010070 con 0x7f884c154a80 2024-03-04T12:12:00.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.861+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f884c0cd2b0 con 0x7f884c154a80 2024-03-04T12:12:00.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.862+0000 7f8842ffd640 1 -- 172.21.15.148:0/1913238265 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f8844058c90 con 0x7f884c154a80 2024-03-04T12:12:00.864 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:00.864 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 msgr2=0x7f882c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 0x7f882c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c009060 comp rx=0 tx=0).stop 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 msgr2=0x7f884c160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8844007f20 tx=0x7f884400d560 comp rx=0 tx=0).stop 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 shutdown_connections 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f882c06b4d0 0x7f882c06d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f884c154a80 0x7f884c160790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f884c0cf650 0x7f884c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 --2- 172.21.15.148:0/1913238265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f884c070de0 0x7f884c1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 >> 172.21.15.148:0/1913238265 conn(0x7f884c06f5f0 msgr2=0x7f884c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 shutdown_connections 2024-03-04T12:12:00.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.865+0000 7f8850a59640 1 -- 172.21.15.148:0/1913238265 wait complete. 2024-03-04T12:12:00.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:00.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.990+0000 7fef5d052640 1 Processor -- start 2024-03-04T12:12:00.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.990+0000 7fef5d052640 1 -- start start 2024-03-04T12:12:00.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5d052640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef580ce0a0 0x7fef581618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5d052640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef58161de0 0x7fef580cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5d052640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580cf200 con 0x7fef5815dcd0 2024-03-04T12:12:00.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5d052640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580cc0c0 con 0x7fef58161de0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5d052640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58162230 con 0x7fef580ce0a0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef58161de0 0x7fef580cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef56d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef5815dcd0 0x7fef580cfeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40440/0 (socket says 172.21.15.148:40440) 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef56d9d640 1 -- 172.21.15.148:0/1719716310 learned_addr learned my addr 172.21.15.148:0/1719716310 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5759e640 1 -- 172.21.15.148:0/1719716310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef580ce0a0 msgr2=0x7fef581618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5759e640 1 --2- 172.21.15.148:0/1719716310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef580ce0a0 0x7fef581618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5759e640 1 -- 172.21.15.148:0/1719716310 >> v1:172.21.15.148:6789/0 conn(0x7fef5815dcd0 legacy=0x7fef580cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef55d9b640 1 -- 172.21.15.148:0/1719716310 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001764781 0 0) 0x7fef580cf200 con 0x7fef5815dcd0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.991+0000 7fef5759e640 1 -- 172.21.15.148:0/1719716310 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580ce680 con 0x7fef58161de0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.992+0000 7fef5759e640 1 --2- 172.21.15.148:0/1719716310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef580cc510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef500097a0 tx=0x7fef5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c808fda26c7da8e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.992+0000 7fef55d9b640 1 -- 172.21.15.148:0/1719716310 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50035020 con 0x7fef58161de0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.992+0000 7fef55d9b640 1 -- 172.21.15.148:0/1719716310 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef50031830 con 0x7fef58161de0 2024-03-04T12:12:00.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.992+0000 7fef55d9b640 1 -- 172.21.15.148:0/1719716310 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50031c00 con 0x7fef58161de0 2024-03-04T12:12:00.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 -- 172.21.15.148:0/1719716310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 msgr2=0x7fef580cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 --2- 172.21.15.148:0/1719716310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef580cc510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef500097a0 tx=0x7fef5002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 -- 172.21.15.148:0/1719716310 shutdown_connections 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 --2- 172.21.15.148:0/1719716310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef580cc510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 --2- 172.21.15.148:0/1719716310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef580ce0a0 0x7fef581618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 -- 172.21.15.148:0/1719716310 >> 172.21.15.148:0/1719716310 conn(0x7fef580b3480 msgr2=0x7fef580b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 -- 172.21.15.148:0/1719716310 shutdown_connections 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.993+0000 7fef5d052640 1 -- 172.21.15.148:0/1719716310 wait complete. 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.994+0000 7fef5d052640 1 Processor -- start 2024-03-04T12:12:00.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.994+0000 7fef5d052640 1 -- start start 2024-03-04T12:12:00.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef580ce0a0 0x7fef580b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef5815dcd0 0x7fef581f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580cef20 con 0x7fef58161de0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580cec40 con 0x7fef580ce0a0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5d052640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5806bff0 con 0x7fef5815dcd0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41514/0 (socket says 172.21.15.148:41514) 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 -- 172.21.15.148:0/2507238294 learned_addr learned my addr 172.21.15.148:0/2507238294 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5659c640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef5815dcd0 0x7fef581f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef56d9d640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef580ce0a0 0x7fef580b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 -- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef5815dcd0 msgr2=0x7fef581f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef5815dcd0 0x7fef581f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 -- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef580ce0a0 msgr2=0x7fef580b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef580ce0a0 0x7fef580b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.995+0000 7fef5759e640 1 -- 172.21.15.148:0/2507238294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580b6f40 con 0x7fef58161de0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef5759e640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef500098c0 tx=0x7fef500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5003b940 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5003bae0 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef500453f0 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef580b7220 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.996+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef58151a20 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.997+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef581f72a0 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.997+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fef50038020 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.997+0000 7fef3f7fe640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 0x7fef4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.998+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fef500ac250 con 0x7fef58161de0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.998+0000 7fef56d9d640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 0x7fef4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:00.998+0000 7fef56d9d640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 0x7fef4406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef480012e0 tx=0x7fef48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:01.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.002+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef50040070 con 0x7fef58161de0 2024-03-04T12:12:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.207+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef580b78b0 con 0x7fef58161de0 2024-03-04T12:12:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.207+0000 7fef3f7fe640 1 -- 172.21.15.148:0/2507238294 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fef5007ae20 con 0x7fef58161de0 2024-03-04T12:12:01.209 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:01.209 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:11:59.272589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T12:12:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 6 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 msgr2=0x7fef4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 0x7fef4406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef480012e0 tx=0x7fef48009040 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 msgr2=0x7fef581514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef500098c0 tx=0x7fef500046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 shutdown_connections 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef4406b4d0 0x7fef4406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58161de0 0x7fef581514e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef5815dcd0 0x7fef581f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 --2- 172.21.15.148:0/2507238294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef580ce0a0 0x7fef580b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 >> 172.21.15.148:0/2507238294 conn(0x7fef580b3480 msgr2=0x7fef580cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 shutdown_connections 2024-03-04T12:12:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.209+0000 7fef5d052640 1 -- 172.21.15.148:0/2507238294 wait complete. 2024-03-04T12:12:01.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:01.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.321+0000 7fde3ac59640 1 Processor -- start 2024-03-04T12:12:01.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.321+0000 7fde3ac59640 1 -- start start 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde3ac59640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 0x7fde340b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde3ac59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde340b5470 0x7fde340b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde3ac59640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3415e1e0 con 0x7fde3415de00 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde3ac59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340b98c0 con 0x7fde340b5470 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde3ac59640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340b32c0 con 0x7fde340b2e60 2024-03-04T12:12:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde340b5470 0x7fde340b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde389f5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fde3415de00 0x7fde340b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40446/0 (socket says 172.21.15.148:40446) 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde340b5470 0x7fde340b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41530/0 (socket says 172.21.15.148:41530) 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde389f5640 1 -- 172.21.15.148:0/1817208709 learned_addr learned my addr 172.21.15.148:0/1817208709 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde33fff640 1 --2- 172.21.15.148:0/1817208709 >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 0x7fde340b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 -- 172.21.15.148:0/1817208709 >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 msgr2=0x7fde340b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 --2- 172.21.15.148:0/1817208709 >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 0x7fde340b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 -- 172.21.15.148:0/1817208709 >> v1:172.21.15.148:6789/0 conn(0x7fde3415de00 legacy=0x7fde340b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde391f6640 1 -- 172.21.15.148:0/1817208709 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340d1c10 con 0x7fde340b5470 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.322+0000 7fde33fff640 1 --2- 172.21.15.148:0/1817208709 >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 0x7fde340b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.323+0000 7fde391f6640 1 --2- 172.21.15.148:0/1817208709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b5470 0x7fde340b5900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde34161830 tx=0x7fde2c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cd3129db790707 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.323+0000 7fde337fe640 1 -- 172.21.15.148:0/1817208709 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c039070 con 0x7fde340b5470 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.323+0000 7fde337fe640 1 -- 172.21.15.148:0/1817208709 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c02cc80 con 0x7fde340b5470 2024-03-04T12:12:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.323+0000 7fde337fe640 1 -- 172.21.15.148:0/1817208709 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c034660 con 0x7fde340b5470 2024-03-04T12:12:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 -- 172.21.15.148:0/1817208709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b5470 msgr2=0x7fde340b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 --2- 172.21.15.148:0/1817208709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b5470 0x7fde340b5900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde34161830 tx=0x7fde2c02bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 -- 172.21.15.148:0/1817208709 shutdown_connections 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 --2- 172.21.15.148:0/1817208709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b5470 0x7fde340b5900 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 --2- 172.21.15.148:0/1817208709 >> v2:172.21.15.191:3300/0 conn(0x7fde340b2e60 0x7fde340b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 -- 172.21.15.148:0/1817208709 >> 172.21.15.148:0/1817208709 conn(0x7fde340d0470 msgr2=0x7fde340d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 -- 172.21.15.148:0/1817208709 shutdown_connections 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.324+0000 7fde3ac59640 1 -- 172.21.15.148:0/1817208709 wait complete. 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.325+0000 7fde3ac59640 1 Processor -- start 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.325+0000 7fde3ac59640 1 -- start start 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 0x7fde3415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 0x7fde3415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340d21d0 con 0x7fde340b2e60 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340d2820 con 0x7fde340b5470 2024-03-04T12:12:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340d24b0 con 0x7fde3415de00 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41540/0 (socket says 172.21.15.148:41540) 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 -- 172.21.15.148:0/3180618431 learned_addr learned my addr 172.21.15.148:0/3180618431 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde33fff640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 0x7fde3415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde391f6640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 0x7fde3415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 -- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 msgr2=0x7fde3415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 0x7fde3415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 -- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 msgr2=0x7fde3415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 0x7fde3415e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 -- 172.21.15.148:0/3180618431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340cbb90 con 0x7fde340b2e60 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde391f6640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 0x7fde3415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde33fff640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 0x7fde3415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde389f5640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c3f0 tx=0x7fde2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24017070 con 0x7fde340b2e60 2024-03-04T12:12:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24005ce0 con 0x7fde340b2e60 2024-03-04T12:12:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde340cbe70 con 0x7fde340b2e60 2024-03-04T12:12:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240072f0 con 0x7fde340b2e60 2024-03-04T12:12:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.326+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde340cc430 con 0x7fde340b2e60 2024-03-04T12:12:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.327+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde34074f60 con 0x7fde340b2e60 2024-03-04T12:12:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.328+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fde2401a070 con 0x7fde340b2e60 2024-03-04T12:12:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.328+0000 7fde317fa640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 0x7fde2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.328+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fde240898e0 con 0x7fde340b2e60 2024-03-04T12:12:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.328+0000 7fde33fff640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 0x7fde2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.329+0000 7fde33fff640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 0x7fde2006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde28006fd0 tx=0x7fde28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:01.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.332+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde24058420 con 0x7fde340b2e60 2024-03-04T12:12:01.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.503+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde340b9340 con 0x7fde340b2e60 2024-03-04T12:12:01.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.504+0000 7fde317fa640 1 -- 172.21.15.148:0/3180618431 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7fde24057d20 con 0x7fde340b2e60 2024-03-04T12:12:01.506 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:01.506 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 msgr2=0x7fde2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 0x7fde2006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde28006fd0 tx=0x7fde28008040 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 msgr2=0x7fde34158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c3f0 tx=0x7fde2400c8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 shutdown_connections 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fde2006b500 0x7fde2006d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde3415de00 0x7fde3415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde340b5470 0x7fde3415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 --2- 172.21.15.148:0/3180618431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde340b2e60 0x7fde34158150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 >> 172.21.15.148:0/3180618431 conn(0x7fde340d0470 msgr2=0x7fde3406c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 shutdown_connections 2024-03-04T12:12:01.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:01.506+0000 7fde3ac59640 1 -- 172.21.15.148:0/3180618431 wait complete. 2024-03-04T12:12:01.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T12:11:54.225836+0000', 'modified': '2024-03-04T12:11:59.272589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/2190957164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6833', 'nonce': 2190957164}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 2190957164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-04T12:12:01.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T12:12:01.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T12:12:02.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.023+0000 7fadcd0a4640 1 Processor -- start 2024-03-04T12:12:02.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.023+0000 7fadcd0a4640 1 -- start start 2024-03-04T12:12:02.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadcd0a4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fadc815aa90 0x7fadc806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadcd0a4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadc806f470 0x7fadc80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadcd0a4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc806f9f0 con 0x7fadc80cf110 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadcd0a4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc8070600 con 0x7fadc806f470 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadcd0a4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc80cf560 con 0x7fadc815aa90 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadc659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadc806f470 0x7fadc80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadc659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadc806f470 0x7fadc80cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41552/0 (socket says 172.21.15.148:41552) 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadc759e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fadc80cf110 0x7fadc80cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40450/0 (socket says 172.21.15.148:40450) 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.024+0000 7fadc659c640 1 -- 172.21.15.148:0/3358445378 learned_addr learned my addr 172.21.15.148:0/3358445378 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/3358445378 >> v2:172.21.15.191:3300/0 conn(0x7fadc815aa90 0x7fadc806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc659c640 1 -- 172.21.15.148:0/3358445378 >> v2:172.21.15.191:3300/0 conn(0x7fadc815aa90 msgr2=0x7fadc806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc659c640 1 --2- 172.21.15.148:0/3358445378 >> v2:172.21.15.191:3300/0 conn(0x7fadc815aa90 0x7fadc806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc659c640 1 -- 172.21.15.148:0/3358445378 >> v1:172.21.15.148:6789/0 conn(0x7fadc80cf110 legacy=0x7fadc80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc659c640 1 -- 172.21.15.148:0/3358445378 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc806fcd0 con 0x7fadc806f470 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc5d9b640 1 -- 172.21.15.148:0/3358445378 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348172665 0 0) 0x7fadc806f9f0 con 0x7fadc80cf110 2024-03-04T12:12:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc659c640 1 --2- 172.21.15.148:0/3358445378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc806f470 0x7fadc80cebd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadc804f710 tx=0x7fadbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74aa4774dbfcc4f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.025+0000 7fadc5d9b640 1 -- 172.21.15.148:0/3358445378 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc034070 con 0x7fadc806f470 2024-03-04T12:12:02.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadc5d9b640 1 -- 172.21.15.148:0/3358445378 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc030860 con 0x7fadc806f470 2024-03-04T12:12:02.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadc5d9b640 1 -- 172.21.15.148:0/3358445378 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc030c30 con 0x7fadc806f470 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 -- 172.21.15.148:0/3358445378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc806f470 msgr2=0x7fadc80cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/3358445378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc806f470 0x7fadc80cebd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadc804f710 tx=0x7fadbc02b9a0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 -- 172.21.15.148:0/3358445378 shutdown_connections 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/3358445378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc806f470 0x7fadc80cebd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/3358445378 >> v2:172.21.15.191:3300/0 conn(0x7fadc815aa90 0x7fadc806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 -- 172.21.15.148:0/3358445378 >> 172.21.15.148:0/3358445378 conn(0x7fadc80d0780 msgr2=0x7fadc80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 -- 172.21.15.148:0/3358445378 shutdown_connections 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.026+0000 7fadcd0a4640 1 -- 172.21.15.148:0/3358445378 wait complete. 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 Processor -- start 2024-03-04T12:12:02.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 -- start start 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc806f470 0x7fadc8156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc815aa90 0x7fadc815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc815ae70 con 0x7fadc80cf110 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8070290 con 0x7fadc815aa90 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadcd0a4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80afda0 con 0x7fadc806f470 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadc659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadc659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41554/0 (socket says 172.21.15.148:41554) 2024-03-04T12:12:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.028+0000 7fadc659c640 1 -- 172.21.15.148:0/4070960858 learned_addr learned my addr 172.21.15.148:0/4070960858 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc806f470 0x7fadc8156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc759e640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc815aa90 0x7fadc815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 -- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc806f470 msgr2=0x7fadc8156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc806f470 0x7fadc8156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 -- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc815aa90 msgr2=0x7fadc815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc815aa90 0x7fadc815d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 -- 172.21.15.148:0/4070960858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc81f7360 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadc659c640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02b9e0 tx=0x7fadbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadaf7fe640 1 -- 172.21.15.148:0/4070960858 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc034040 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadaf7fe640 1 -- 172.21.15.148:0/4070960858 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc0433f0 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc81f7640 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadaf7fe640 1 -- 172.21.15.148:0/4070960858 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0437e0 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.029+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81f7c00 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.030+0000 7fadaf7fe640 1 -- 172.21.15.148:0/4070960858 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fadbc04a060 con 0x7fadc80cf110 2024-03-04T12:12:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.031+0000 7fadaf7fe640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 0x7fadb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.031+0000 7fadaf7fe640 1 -- 172.21.15.148:0/4070960858 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadbc0ac770 con 0x7fadc80cf110 2024-03-04T12:12:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.031+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 0x7fadb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.031+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 0x7fadb406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadb8008e50 tx=0x7fadb80083a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.032+0000 7fadcfd66740 1 Processor -- start 2024-03-04T12:12:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.032+0000 7fadcfd66740 1 -- start start 2024-03-04T12:12:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b37ea0 0x55e830b0ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b38280 0x55e830b3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e830a757c0 con 0x55e830b0b380 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e830a1eb50 con 0x55e830b38280 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e830a71860 con 0x55e830b37ea0 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41570/0 (socket says 172.21.15.148:41570) 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 -- 172.21.15.148:0/1293694775 learned_addr learned my addr 172.21.15.148:0/1293694775 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b37ea0 0x55e830b0ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc659c640 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b38280 0x55e830b3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 -- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b37ea0 msgr2=0x55e830b0ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b37ea0 0x55e830b0ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 -- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b38280 msgr2=0x55e830b3ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b38280 0x55e830b3ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 -- 172.21.15.148:0/1293694775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e830a329a0 con 0x55e830b0b380 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.033+0000 7fadc759e640 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadc0001620 tx=0x7fadc0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfa55957028fe89 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadacff9640 1 -- 172.21.15.148:0/1293694775 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc00072d0 con 0x55e830b0b380 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadacff9640 1 -- 172.21.15.148:0/1293694775 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0007470 con 0x55e830b0b380 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadacff9640 1 -- 172.21.15.148:0/1293694775 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc00049e0 con 0x55e830b0b380 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 -- 172.21.15.148:0/1293694775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 msgr2=0x55e830b3b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadc0001620 tx=0x7fadc0001ae0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 -- 172.21.15.148:0/1293694775 shutdown_connections 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b0b380 0x55e830b3b170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b38280 0x55e830b3ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 --2- 172.21.15.148:0/1293694775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b37ea0 0x55e830b0ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 -- 172.21.15.148:0/1293694775 >> 172.21.15.148:0/1293694775 conn(0x55e830b0d0b0 msgr2=0x55e830b55c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 -- 172.21.15.148:0/1293694775 shutdown_connections 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.034+0000 7fadcfd66740 1 -- 172.21.15.148:0/1293694775 wait complete. 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.035+0000 7fadcfd66740 1 -- start start 2024-03-04T12:12:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.035+0000 7fadcfd66740 1 Processor -- start 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 0x55e830b2d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e830b0f750 con 0x55e830b37ea0 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e830a31f90 con 0x55e830b2cd70 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e830b11f90 con 0x55e830b38280 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41580/0 (socket says 172.21.15.148:41580) 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 -- 172.21.15.148:0/3666681253 learned_addr learned my addr 172.21.15.148:0/3666681253 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 10 client.0 _ll_register_callbacks cb 0x55e830a53d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc659c640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc759e640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 0x55e830b2d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 -- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 msgr2=0x55e830b390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 -- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 msgr2=0x55e830b2d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 0x55e830b2d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 -- 172.21.15.148:0/3666681253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e83078a2d0 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc759e640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 0x55e830b2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fadb8010b10 tx=0x7fadb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbdcd518cf12193 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fada5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8022070 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadb800fb50 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc0007470 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb802d050 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 10 client.4425 fetch_fsmap learned FSMap version 6 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.036+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e8308546b0 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0007470 con 0x55e830b37ea0 2024-03-04T12:12:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fadb8030020 con 0x55e830b37ea0 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 10 client.4425 fetch_fsmap finished waiting for FSMap version 6 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 10 client.4425 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 20 client.4425 populate_metadata read hostname 'smithi148' 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b69b10 0x55e830b69ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 4 client.4425 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 --> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e830a82d00 con 0x55e830b69b10 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadc659c640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b69b10 0x55e830b69ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fada5ffb640 10 client.4425 ms_handle_connect on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.037+0000 7fadc659c640 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b69b10 0x55e830b69ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd0983afc35f821 server_cookie=4f282825ee2864c7 in_seq=0 out_seq=0 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fada5ffb640 1 -- 172.21.15.148:0/3666681253 <== mds.0 v2:172.21.15.191:6833/2190957164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e830a82d00 con 0x55e830b69b10 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fada5ffb640 10 client.4425 handle_command_reply: tid=0 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 client.4425 shutdown 2024-03-04T12:12:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 msgr2=0x55e830b359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fadb8010b10 tx=0x7fadb8004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 shutdown_connections 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b69b10 0x55e830b69ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b2cd70 0x55e830b2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fada5ffb640 0 client.4425 ms_handle_reset on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 --2- 172.21.15.148:0/3666681253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b37ea0 0x55e830b359f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 >> 172.21.15.148:0/3666681253 conn(0x55e830b0b380 msgr2=0x55e830b0edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 shutdown_connections 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 1 -- 172.21.15.148:0/3666681253 wait complete. 2024-03-04T12:12:02.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.038+0000 7fadcfd66740 20 client.4425 trim_cache size 0 max 16384 2024-03-04T12:12:02.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 Processor -- start 2024-03-04T12:12:02.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 -- start start 2024-03-04T12:12:02.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830b78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b2c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.054+0000 7fadcfd66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8201440 con 0x55e830b2cd70 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8200620 con 0x55e830b37ea0 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80b4980 con 0x55e830b38280 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41582/0 (socket says 172.21.15.148:41582) 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 -- 172.21.15.148:0/484200261 learned_addr learned my addr 172.21.15.148:0/484200261 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 -- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 msgr2=0x55e830b2c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b2c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 -- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 msgr2=0x55e830b78f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830b78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 -- 172.21.15.148:0/484200261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e830a7aa40 con 0x55e830b2cd70 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fadb80107c0 tx=0x7fadb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2370b902ee9840 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fada7fff640 1 -- 172.21.15.148:0/484200261 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8022040 con 0x55e830b2cd70 2024-03-04T12:12:02.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fada7fff640 1 -- 172.21.15.148:0/484200261 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8016040 con 0x55e830b2cd70 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.055+0000 7fada7fff640 1 -- 172.21.15.148:0/484200261 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb800f790 con 0x55e830b2cd70 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- 172.21.15.148:0/484200261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 msgr2=0x55e830b3ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fadb80107c0 tx=0x7fadb8017000 comp rx=0 tx=0).stop 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- 172.21.15.148:0/484200261 shutdown_connections 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b2c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830b78f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 --2- 172.21.15.148:0/484200261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830b3ad00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- 172.21.15.148:0/484200261 >> 172.21.15.148:0/484200261 conn(0x55e830b0b380 msgr2=0x55e830b35800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- 172.21.15.148:0/484200261 shutdown_connections 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- 172.21.15.148:0/484200261 wait complete. 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 -- start start 2024-03-04T12:12:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.056+0000 7fadcfd66740 1 Processor -- start 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830bb76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b31850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e830b3c590 con 0x55e830b2cd70 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8200020 con 0x55e830b37ea0 2024-03-04T12:12:02.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e830b3bdd0 con 0x55e830b38280 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadcfd66740 10 client.0 _ll_register_callbacks cb 0x7fadc0007d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41596/0 (socket says 172.21.15.148:41596) 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 -- 172.21.15.148:0/852454362 learned_addr learned my addr 172.21.15.148:0/852454362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc659c640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830bb76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc759e640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b31850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 -- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 msgr2=0x55e830b31850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b31850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 -- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 msgr2=0x55e830bb76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830bb76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 -- 172.21.15.148:0/852454362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e830b31d90 con 0x55e830b2cd70 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fadc6d9d640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadb80085a0 tx=0x7fadb8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2320d7dbf5f4a0ed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fada5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:12:02.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.057+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8022040 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadb800fb50 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadb8016040 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb804a050 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 10 client.4431 fetch_fsmap learned FSMap version 6 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e830b307f0 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8016040 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fadb804d020 con 0x55e830b2cd70 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 10 client.4431 fetch_fsmap finished waiting for FSMap version 6 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 10 client.4431 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 20 client.4431 populate_metadata read hostname 'smithi148' 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b10f80 0x55e830bc4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 4 client.4431 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 --> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e830a7d050 con 0x55e830b10f80 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadc659c640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b10f80 0x55e830bc4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fada5ffb640 10 client.4431 ms_handle_connect on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.058+0000 7fadc659c640 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b10f80 0x55e830bc4200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e998381e6c6961 server_cookie=cca96382d6e62f4a in_seq=0 out_seq=0 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fada5ffb640 1 -- 172.21.15.148:0/852454362 <== mds.0 v2:172.21.15.191:6833/2190957164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e830a7d050 con 0x55e830b10f80 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fada5ffb640 10 client.4431 handle_command_reply: tid=0 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 client.4431 shutdown 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 msgr2=0x55e830bc0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadb80085a0 tx=0x7fadb8014000 comp rx=0 tx=0).stop 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 shutdown_connections 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x55e830b10f80 0x55e830bc4200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e830b38280 0x55e830b31850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e830b37ea0 0x55e830bb76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fada5ffb640 0 client.4431 ms_handle_reset on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 --2- 172.21.15.148:0/852454362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e830b2cd70 0x55e830bc0c10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 >> 172.21.15.148:0/852454362 conn(0x55e830b0b380 msgr2=0x55e830b78d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 shutdown_connections 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 1 -- 172.21.15.148:0/852454362 wait complete. 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcfd66740 20 client.4431 trim_cache size 0 max 16384 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "d4b3ed31-a94c-42d0-968e-1d6a4e199a28", 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4288, 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-04T12:12:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 7.1633904050000003, 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 6, 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 24, 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 24, 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 13.681024288 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-04T12:12:02.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 msgr2=0x7fadb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 0x7fadb406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadb8008e50 tx=0x7fadb80083a0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 msgr2=0x7fadc806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02b9e0 tx=0x7fadbc0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 shutdown_connections 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fadb406b500 0x7fadb406d9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc815aa90 0x7fadc815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadc80cf110 0x7fadc806b4a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 --2- 172.21.15.148:0/4070960858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc806f470 0x7fadc8156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 >> 172.21.15.148:0/4070960858 conn(0x7fadc80d0780 msgr2=0x7fadc80cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 shutdown_connections 2024-03-04T12:12:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.059+0000 7fadcd0a4640 1 -- 172.21.15.148:0/4070960858 wait complete. 2024-03-04T12:12:02.071 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T12:12:02.071 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:02.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:12:02.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.170+0000 7fc4a89c4640 1 Processor -- start 2024-03-04T12:12:02.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.170+0000 7fc4a89c4640 1 -- start start 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.171+0000 7fc4a89c4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4a40cfae0 0x7fc4a40cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.171+0000 7fc4a89c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc4a40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.171+0000 7fc4a89c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a40cb300 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.171+0000 7fc4a89c4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a406ae50 con 0x7fc4a40cfae0 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.171+0000 7fc4a89c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406b130 con 0x7fc4a40d1290 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a1d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc4a40cc8e0 0x7fc4a40cfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40460/0 (socket says 172.21.15.148:40460) 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a1d9b640 1 -- 172.21.15.148:0/4263273355 learned_addr learned my addr 172.21.15.148:0/4263273355 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a259c640 1 --2- 172.21.15.148:0/4263273355 >> v2:172.21.15.191:3300/0 conn(0x7fc4a40cfae0 0x7fc4a40cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc4a40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916328528 0 0) 0x7fc4a40cb300 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc48c003520 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122374301 0 0) 0x7fc48c003520 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 msgr2=0x7fc4a40cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc4a40cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 >> v2:172.21.15.191:3300/0 conn(0x7fc4a40cfae0 msgr2=0x7fc4a40cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 --2- 172.21.15.148:0/4263273355 >> v2:172.21.15.191:3300/0 conn(0x7fc4a40cfae0 0x7fc4a40cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a406b410 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fc4980019f0 con 0x7fc4a40cc8e0 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 >> v1:172.21.15.148:6789/0 conn(0x7fc4a40cc8e0 legacy=0x7fc4a40cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc48c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc48c003f30 0x7fc48c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc48c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406ae50 con 0x7fc48c004d40 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406b130 con 0x7fc48c003f30 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c00b9c0 con 0x7fc4a40d1290 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc48c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a259c640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc48c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a1d9b640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc48c003f30 0x7fc48c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 -- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 msgr2=0x7fc48c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a40d1290 0x7fc48c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 -- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc48c003f30 msgr2=0x7fc48c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.172+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc48c003f30 0x7fc48c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.173+0000 7fc4a2d9d640 1 -- 172.21.15.148:0/4263273355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c003520 con 0x7fc48c004d40 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.173+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc48c0051c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc494009ac0 tx=0x7fc48c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa2ddc4ea934a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.173+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494039070 con 0x7fc48c004d40 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.173+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49402c6b0 con 0x7fc48c004d40 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.173+0000 7fc4a159a640 1 -- 172.21.15.148:0/4263273355 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494009690 con 0x7fc48c004d40 2024-03-04T12:12:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 -- 172.21.15.148:0/4263273355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 msgr2=0x7fc48c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/4263273355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc48c0051c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc494009ac0 tx=0x7fc48c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a2d9d640 1 -- 172.21.15.148:0/4263273355 reap_dead start 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 -- 172.21.15.148:0/4263273355 shutdown_connections 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 -- 172.21.15.148:0/4263273355 >> 172.21.15.148:0/4263273355 conn(0x7fc4a40d0770 msgr2=0x7fc4a40d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 -- 172.21.15.148:0/4263273355 shutdown_connections 2024-03-04T12:12:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.174+0000 7fc4a89c4640 1 -- 172.21.15.148:0/4263273355 wait complete. 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.175+0000 7fc4a89c4640 1 Processor -- start 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.175+0000 7fc4a89c4640 1 -- start start 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a40b2850 0x7fc4a40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a41600f0 0x7fc4a4160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406b9d0 con 0x7fc48c004d40 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40ce440 con 0x7fc4a40b2850 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a89c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cd440 con 0x7fc4a41600f0 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41604/0 (socket says 172.21.15.148:41604) 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 -- 172.21.15.148:0/1500268002 learned_addr learned my addr 172.21.15.148:0/1500268002 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a1d9b640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a40b2850 0x7fc4a40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a2d9d640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a41600f0 0x7fc4a4160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 -- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a41600f0 msgr2=0x7fc4a4160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a41600f0 0x7fc4a4160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 -- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a40b2850 msgr2=0x7fc4a40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a40b2850 0x7fc4a40b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 -- 172.21.15.148:0/1500268002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a40c94c0 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.176+0000 7fc4a259c640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc49c011ad0 tx=0x7fc49c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.177+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00fb50 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.177+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00fcf0 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.177+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00fea0 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.177+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a40c97a0 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.177+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a415e210 con 0x7fc48c004d40 2024-03-04T12:12:02.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.178+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a406cd70 con 0x7fc48c004d40 2024-03-04T12:12:02.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.178+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc49c012040 con 0x7fc48c004d40 2024-03-04T12:12:02.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.178+0000 7fc492ffd640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 0x7fc48406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:02.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.179+0000 7fc4a1d9b640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 0x7fc48406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:02.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.179+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc49c08aab0 con 0x7fc48c004d40 2024-03-04T12:12:02.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.179+0000 7fc4a1d9b640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 0x7fc48406d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40c6e10 tx=0x7fc498006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:02.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.183+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc49c019030 con 0x7fc48c004d40 2024-03-04T12:12:02.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.388+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4a415bcd0 con 0x7fc48c004d40 2024-03-04T12:12:02.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.388+0000 7fc492ffd640 1 -- 172.21.15.148:0/1500268002 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc49c059680 con 0x7fc48c004d40 2024-03-04T12:12:02.389 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T12:12:02.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 msgr2=0x7fc48406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 0x7fc48406d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40c6e10 tx=0x7fc498006040 comp rx=0 tx=0).stop 2024-03-04T12:12:02.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 msgr2=0x7fc4a4157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:02.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc49c011ad0 tx=0x7fc49c011f90 comp rx=0 tx=0).stop 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 shutdown_connections 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc48406b500 0x7fc48406d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4a41600f0 0x7fc4a4160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a40b2850 0x7fc4a40b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 --2- 172.21.15.148:0/1500268002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc48c004d40 0x7fc4a4157ea0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 >> 172.21.15.148:0/1500268002 conn(0x7fc4a40d0770 msgr2=0x7fc4a415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 shutdown_connections 2024-03-04T12:12:02.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:02.390+0000 7fc4a89c4640 1 -- 172.21.15.148:0/1500268002 wait complete. 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:12:02.402 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T12:12:02.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.402 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-04T12:12:02.417 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:02.417 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:02.417 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42719sec preferred_lft 42719sec 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-04T12:12:02.418 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:02.419 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T12:12:02.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.419 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:02.419 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:02.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-03-04T12:12:02.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-03-04T12:12:02.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-03-04T12:12:02.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T12:12:02.420 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:02.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.580 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:12:02.607 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-04T12:12:02.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.609 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:12:02.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.679 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:02.679 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:02.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:12:02.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:12:02.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:12:02.680 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:02.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.951 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:12:02.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:02.966 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-04T12:12:03.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.021 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:03.021 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:03.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:03.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-04T12:12:03.021 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:03.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-04T12:12:03.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-04T12:12:03.109 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:03.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.212 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:03.212 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:03.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-04T12:12:03.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T12:12:03.212 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:03.254 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-04T12:12:03.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:03.254 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:03.309 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T12:12:03.309 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:03.309 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:03.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-04T12:12:03.432 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T12:12:03.488 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-04T12:12:03.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-04T12:12:03.490 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:03.491 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:03.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.491 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-04T12:12:03.544 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-04T12:12:03.544 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-04T12:12:03.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-04T12:12:03.616 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T12:12:03.671 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-04T12:12:03.671 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-04T12:12:03.671 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-04T12:12:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-04T12:12:03.673 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:03.674 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:03.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.674 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-04T12:12:03.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:2024-03-04T12:12:03.676+0000 7f8215a6d480 -1 init, newargv = 0x559a43d48ab0 newargc=15 2024-03-04T12:12:03.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[45650]: starting ceph client 2024-03-04T12:12:03.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[45650]: starting fuse 2024-03-04T12:12:03.692 INFO:teuthology.orchestra.run.smithi148.stdout:43 2024-03-04T12:12:03.692 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-04T12:12:03.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-03-04T12:12:03.692 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-04T12:12:03.692 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-04T12:12:03.692 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-04T12:12:03.692 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-04T12:12:03.693 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.admin")) 2024-03-04T12:12:03.694 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-04T12:12:03.814 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.admin.45650.asok 2024-03-04T12:12:03.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-04T12:12:03.825 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.45650.asok 2024-03-04T12:12:03.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.45650.asok status 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "717ce3cce8d0166e4e577faa8004e6f9cb4128c0", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "admin", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "45650", 2024-03-04T12:12:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4443, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4443 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2274361058 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:12:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2274361058 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4443 192.168.0.1:0/2274361058", 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/2274361058", 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 6, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-04T12:12:03.965 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-04T12:12:03.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.972 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-04T12:12:03.972 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:03.987 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-04T12:12:03.987 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:03.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:03.987 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:04.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:04.056 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-04T12:12:04.126 INFO:teuthology.orchestra.run.smithi148.stdout:# file: . 2024-03-04T12:12:04.126 INFO:teuthology.orchestra.run.smithi148.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-04T12:12:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:04.126 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:04.126 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:04.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.322+0000 7f9aa431b640 1 Processor -- start 2024-03-04T12:12:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.322+0000 7f9aa431b640 1 -- start start 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa431b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a9c0cfd60 0x7f9a9c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa431b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c0d0140 con 0x7f9a9c0cf0d0 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa431b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c15c280 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa431b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c15cbb0 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa20b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a9c0cfd60 0x7f9a9c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa28b8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9a9c0cf0d0 0x7f9a9c15f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52232/0 (socket says 172.21.15.148:52232) 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa28b8640 1 -- 172.21.15.148:0/3318436187 learned_addr learned my addr 172.21.15.148:0/3318436187 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa20b7640 1 -- 172.21.15.148:0/3318436187 >> v1:172.21.15.191:6789/0 conn(0x7f9a9c06d3b0 legacy=0x7f9a9c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.323+0000 7f9aa20b7640 1 -- 172.21.15.148:0/3318436187 >> v1:172.21.15.148:6789/0 conn(0x7f9a9c0cf0d0 legacy=0x7f9a9c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa10b5640 1 -- 172.21.15.148:0/3318436187 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605568092 0 0) 0x7f9a9c0d0140 con 0x7f9a9c0cf0d0 2024-03-04T12:12:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa20b7640 1 -- 172.21.15.148:0/3318436187 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c15c560 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa20b7640 1 --2- 172.21.15.148:0/3318436187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c0cbdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c15cd30 tx=0x7f9a8c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ee39012a0959fa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa10b5640 1 -- 172.21.15.148:0/3318436187 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c034070 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa10b5640 1 -- 172.21.15.148:0/3318436187 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c030610 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.324+0000 7f9aa10b5640 1 -- 172.21.15.148:0/3318436187 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0309e0 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 -- 172.21.15.148:0/3318436187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c0cfd60 msgr2=0x7f9a9c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 --2- 172.21.15.148:0/3318436187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c0cbdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c15cd30 tx=0x7f9a8c02b960 comp rx=0 tx=0).stop 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 -- 172.21.15.148:0/3318436187 shutdown_connections 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 --2- 172.21.15.148:0/3318436187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c0cbdf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 -- 172.21.15.148:0/3318436187 >> 172.21.15.148:0/3318436187 conn(0x7f9a9c06acd0 msgr2=0x7f9a9c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 -- 172.21.15.148:0/3318436187 shutdown_connections 2024-03-04T12:12:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.325+0000 7f9aa431b640 1 -- 172.21.15.148:0/3318436187 wait complete. 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.326+0000 7f9aa431b640 1 Processor -- start 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.326+0000 7f9aa431b640 1 -- start start 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9c0cf0d0 0x7f9a9c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c1f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0cc330 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c15bf30 con 0x7f9a9c0cfd60 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa431b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0cc4b0 con 0x7f9a9c0cf0d0 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa20b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa20b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48172/0 (socket says 172.21.15.148:48172) 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa20b7640 1 -- 172.21.15.148:0/4284179567 learned_addr learned my addr 172.21.15.148:0/4284179567 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa18b6640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9c0cf0d0 0x7f9a9c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.327+0000 7f9aa28b8640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c1f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 -- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9c0cf0d0 msgr2=0x7f9a9c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9c0cf0d0 0x7f9a9c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 -- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0cfd60 msgr2=0x7f9a9c1f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c1f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 -- 172.21.15.148:0/4284179567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c1f2020 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa20b7640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009890 tx=0x7f9a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c034040 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c03ace0 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c1f2300 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c044630 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.328+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c0a9360 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.329+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c074f60 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.329+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9a8c037070 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.330+0000 7f9a92ffd640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 0x7f9a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.330+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9a8c0ac500 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.330+0000 7f9aa18b6640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 0x7f9a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.330+0000 7f9aa18b6640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 0x7f9a8006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c156480 tx=0x7f9a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.334+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a8c03f070 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.506+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a9c158ca0 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.508+0000 7f9a92ffd640 1 -- 172.21.15.148:0/4284179567 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f9a8c07b0d0 con 0x7f9a9c06d3b0 2024-03-04T12:12:04.509 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:04.509 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:04.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 msgr2=0x7f9a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 0x7f9a8006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c156480 tx=0x7f9a98009040 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 msgr2=0x7f9a9c0a8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009890 tx=0x7f9a8c0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 shutdown_connections 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9a8006b4d0 0x7f9a8006d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0cfd60 0x7f9a9c1f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9c0cf0d0 0x7f9a9c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 --2- 172.21.15.148:0/4284179567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9c06d3b0 0x7f9a9c0a8e20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 >> 172.21.15.148:0/4284179567 conn(0x7f9a9c06acd0 msgr2=0x7f9a9c15ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 shutdown_connections 2024-03-04T12:12:04.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.510+0000 7f9aa431b640 1 -- 172.21.15.148:0/4284179567 wait complete. 2024-03-04T12:12:04.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:04.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.626+0000 7f73b1a43640 1 Processor -- start 2024-03-04T12:12:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.626+0000 7f73b1a43640 1 -- start start 2024-03-04T12:12:04.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.626+0000 7f73b1a43640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f73ac15a810 0x7f73ac0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.626+0000 7f73b1a43640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73ac06d3c0 0x7f73ac0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.626+0000 7f73b1a43640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac0cbfb0 0x7f73ac15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73b1a43640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac06d8a0 con 0x7f73ac06d3c0 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73b1a43640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac06db80 con 0x7f73ac15a810 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73b1a43640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06de60 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73ac06d3c0 0x7f73ac0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73ac06d3c0 0x7f73ac0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48184/0 (socket says 172.21.15.148:48184) 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 -- 172.21.15.148:0/1044394241 learned_addr learned my addr 172.21.15.148:0/1044394241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73aaffd640 1 --2- 172.21.15.148:0/1044394241 >> v2:172.21.15.191:3300/0 conn(0x7f73ac15a810 0x7f73ac0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73ab7fe640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac0cbfb0 0x7f73ac15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 -- 172.21.15.148:0/1044394241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac0cbfb0 msgr2=0x7f73ac15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac0cbfb0 0x7f73ac15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 -- 172.21.15.148:0/1044394241 >> v2:172.21.15.191:3300/0 conn(0x7f73ac15a810 msgr2=0x7f73ac0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 --2- 172.21.15.148:0/1044394241 >> v2:172.21.15.191:3300/0 conn(0x7f73ac15a810 0x7f73ac0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.627+0000 7f73a3fff640 1 -- 172.21.15.148:0/1044394241 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac06e140 con 0x7f73ac06d3c0 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73a3fff640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac06d3c0 0x7f73ac0b46d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ac06e560 tx=0x7f739402fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49be880ad0b204c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73aa7fc640 1 -- 172.21.15.148:0/1044394241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739403d070 con 0x7f73ac06d3c0 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73aa7fc640 1 -- 172.21.15.148:0/1044394241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394030ae0 con 0x7f73ac06d3c0 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73aa7fc640 1 -- 172.21.15.148:0/1044394241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394038440 con 0x7f73ac06d3c0 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73b1a43640 1 -- 172.21.15.148:0/1044394241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac06d3c0 msgr2=0x7f73ac0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.628+0000 7f73b1a43640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac06d3c0 0x7f73ac0b46d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ac06e560 tx=0x7f739402fd30 comp rx=0 tx=0).stop 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 -- 172.21.15.148:0/1044394241 shutdown_connections 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac0cbfb0 0x7f73ac15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 --2- 172.21.15.148:0/1044394241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac06d3c0 0x7f73ac0b46d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 --2- 172.21.15.148:0/1044394241 >> v2:172.21.15.191:3300/0 conn(0x7f73ac15a810 0x7f73ac0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 -- 172.21.15.148:0/1044394241 >> 172.21.15.148:0/1044394241 conn(0x7f73ac06acd0 msgr2=0x7f73ac06b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 -- 172.21.15.148:0/1044394241 shutdown_connections 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.629+0000 7f73b1a43640 1 -- 172.21.15.148:0/1044394241 wait complete. 2024-03-04T12:12:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 Processor -- start 2024-03-04T12:12:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 -- start start 2024-03-04T12:12:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73ac06d3c0 0x7f73ac155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac15a810 0x7f73ac071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15b130 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15e5d0 con 0x7f73ac06d3c0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73b1a43640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15b2b0 con 0x7f73ac15a810 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.630+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48192/0 (socket says 172.21.15.148:48192) 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 -- 172.21.15.148:0/3083379035 learned_addr learned my addr 172.21.15.148:0/3083379035 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73aaffd640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73ac06d3c0 0x7f73ac155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73ab7fe640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac15a810 0x7f73ac071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 -- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac15a810 msgr2=0x7f73ac071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac15a810 0x7f73ac071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 -- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73ac06d3c0 msgr2=0x7f73ac155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73ac06d3c0 0x7f73ac155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 -- 172.21.15.148:0/3083379035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0a8820 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a3fff640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f739402fd70 tx=0x7f73940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739403d040 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394048070 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac0a8b00 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394041670 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.631+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac0a90c0 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.632+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac0d0590 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.633+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7394041810 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.633+0000 7f73a8ff9640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 0x7f738c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.633+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f73940b0930 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.633+0000 7f73aaffd640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 0x7f738c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.634+0000 7f73aaffd640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 0x7f738c06d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f739c007920 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.637+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73ac0d0590 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.808+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73ac074f60 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.809+0000 7f73a8ff9640 1 -- 172.21.15.148:0/3083379035 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f739407eeb0 con 0x7f73ac0cbfb0 2024-03-04T12:12:04.810 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 msgr2=0x7f738c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 0x7f738c06d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f739c007920 tx=0x7f739c008040 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 msgr2=0x7f73ac15d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f739402fd70 tx=0x7f73940046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 shutdown_connections 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f738c06b4d0 0x7f738c06d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73ac15a810 0x7f73ac071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73ac0cbfb0 0x7f73ac15d8d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 --2- 172.21.15.148:0/3083379035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73ac06d3c0 0x7f73ac155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 >> 172.21.15.148:0/3083379035 conn(0x7f73ac06acd0 msgr2=0x7f73ac0b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 shutdown_connections 2024-03-04T12:12:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.811+0000 7f73b1a43640 1 -- 172.21.15.148:0/3083379035 wait complete. 2024-03-04T12:12:04.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:04.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 Processor -- start 2024-03-04T12:12:04.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 -- start start 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f069c15df00 0x7f069c06b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f069c06fa80 0x7f069c0cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0cb290 con 0x7f069c0cfa40 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0701d0 con 0x7f069c06fa80 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.926+0000 7f06a2f52640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0704b0 con 0x7f069c15df00 2024-03-04T12:12:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f069bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f069c06fa80 0x7f069c0cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f06a14ef640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f069c0cfa40 0x7f069c0cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52240/0 (socket says 172.21.15.148:52240) 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f06a14ef640 1 -- 172.21.15.148:0/1073501739 learned_addr learned my addr 172.21.15.148:0/1073501739 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f06a0cee640 1 --2- 172.21.15.148:0/1073501739 >> v2:172.21.15.191:3300/0 conn(0x7f069c15df00 0x7f069c06b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f069bfff640 1 -- 172.21.15.148:0/1073501739 >> v2:172.21.15.191:3300/0 conn(0x7f069c15df00 msgr2=0x7f069c06b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f069bfff640 1 --2- 172.21.15.148:0/1073501739 >> v2:172.21.15.191:3300/0 conn(0x7f069c15df00 0x7f069c06b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f069bfff640 1 -- 172.21.15.148:0/1073501739 >> v1:172.21.15.148:6789/0 conn(0x7f069c0cfa40 legacy=0x7f069c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.927+0000 7f069bfff640 1 -- 172.21.15.148:0/1073501739 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0cc8e0 con 0x7f069c06fa80 2024-03-04T12:12:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.928+0000 7f069bfff640 1 --2- 172.21.15.148:0/1073501739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c06fa80 0x7f069c0cad50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f069c0d17d0 tx=0x7f068802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4c38fa63315ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.928+0000 7f069b7fe640 1 -- 172.21.15.148:0/1073501739 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880315a0 con 0x7f069c06fa80 2024-03-04T12:12:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.928+0000 7f069b7fe640 1 -- 172.21.15.148:0/1073501739 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688037070 con 0x7f069c06fa80 2024-03-04T12:12:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.928+0000 7f069b7fe640 1 -- 172.21.15.148:0/1073501739 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688031960 con 0x7f069c06fa80 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 -- 172.21.15.148:0/1073501739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c06fa80 msgr2=0x7f069c0cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1073501739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c06fa80 0x7f069c0cad50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f069c0d17d0 tx=0x7f068802b840 comp rx=0 tx=0).stop 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 -- 172.21.15.148:0/1073501739 shutdown_connections 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1073501739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c06fa80 0x7f069c0cad50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1073501739 >> v2:172.21.15.191:3300/0 conn(0x7f069c15df00 0x7f069c06b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 -- 172.21.15.148:0/1073501739 >> 172.21.15.148:0/1073501739 conn(0x7f069c0b2250 msgr2=0x7f069c0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 -- 172.21.15.148:0/1073501739 shutdown_connections 2024-03-04T12:12:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.929+0000 7f06a2f52640 1 -- 172.21.15.148:0/1073501739 wait complete. 2024-03-04T12:12:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.930+0000 7f06a2f52640 1 Processor -- start 2024-03-04T12:12:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.930+0000 7f06a2f52640 1 -- start start 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f069c06fa80 0x7f069c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c15df00 0x7f069c0c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c06fee0 con 0x7f069c0cfa40 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0ccea0 con 0x7f069c15df00 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a2f52640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0cd3c0 con 0x7f069c06fa80 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48216/0 (socket says 172.21.15.148:48216) 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 -- 172.21.15.148:0/1831118417 learned_addr learned my addr 172.21.15.148:0/1831118417 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a14ef640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c15df00 0x7f069c0c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f06a0cee640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f069c06fa80 0x7f069c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 -- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f069c06fa80 msgr2=0x7f069c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f069c06fa80 0x7f069c0b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 -- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c15df00 msgr2=0x7f069c0c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c15df00 0x7f069c0c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 -- 172.21.15.148:0/1831118417 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0c8510 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.931+0000 7f069bfff640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f06880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.932+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688032630 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.932+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06880327f0 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.932+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688032be0 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.932+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c0c87f0 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.932+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0c8db0 con 0x7f069c0cfa40 2024-03-04T12:12:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.933+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c074f60 con 0x7f069c0cfa40 2024-03-04T12:12:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.933+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f068804c020 con 0x7f069c0cfa40 2024-03-04T12:12:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.933+0000 7f06997fa640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.933+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f068802f020 con 0x7f069c0cfa40 2024-03-04T12:12:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.933+0000 7f06a0cee640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.934+0000 7f06a0cee640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 0x7f068006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0690009aa0 tx=0x7f0690009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:04.937+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0688040070 con 0x7f069c0cfa40 2024-03-04T12:12:05.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.109+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f069c0cdbb0 con 0x7f069c0cfa40 2024-03-04T12:12:05.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.109+0000 7f06997fa640 1 -- 172.21.15.148:0/1831118417 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f068807a5e0 con 0x7f069c0cfa40 2024-03-04T12:12:05.110 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 msgr2=0x7f068006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 0x7f068006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0690009aa0 tx=0x7f0690009190 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 msgr2=0x7f069c0ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f06880046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 shutdown_connections 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f068006b4d0 0x7f068006d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c15df00 0x7f069c0c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069c0cfa40 0x7f069c0ca0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 --2- 172.21.15.148:0/1831118417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f069c06fa80 0x7f069c0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 >> 172.21.15.148:0/1831118417 conn(0x7f069c0b2250 msgr2=0x7f069c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 shutdown_connections 2024-03-04T12:12:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.111+0000 7f06a2f52640 1 -- 172.21.15.148:0/1831118417 wait complete. 2024-03-04T12:12:05.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:05.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.225+0000 7efecc3df640 1 Processor -- start 2024-03-04T12:12:05.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.225+0000 7efecc3df640 1 -- start start 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efecc3df640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efec40cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efecc3df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec4162450 con 0x7efec415da20 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efecc3df640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec40cc2d0 con 0x7efec41616e0 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efecc3df640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40d0000 con 0x7efec40cf660 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efeca17b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efec415da20 0x7efec40ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52252/0 (socket says 172.21.15.148:52252) 2024-03-04T12:12:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efeca17b640 1 -- 172.21.15.148:0/4093426933 learned_addr learned my addr 172.21.15.148:0/4093426933 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efec997a640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efec40cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335750768 0 0) 0x7efec4162450 con 0x7efec415da20 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.226+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb0003520 con 0x7efec415da20 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3714991525 0 0) 0x7efeb0003520 con 0x7efec415da20 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 msgr2=0x7efec40cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efec40cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 >> v1:172.21.15.191:6789/0 conn(0x7efec41616e0 legacy=0x7efec4161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40cc5b0 con 0x7efec415da20 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7efec00019f0 con 0x7efec415da20 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec997a640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efec40cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 >> v1:172.21.15.148:6789/0 conn(0x7efec415da20 legacy=0x7efec40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efeb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeb0003f20 0x7efeb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efeb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb000baa0 con 0x7efeb0004d40 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb000bc50 con 0x7efeb0003f20 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb000be50 con 0x7efec40cf660 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efeb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca17b640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efeb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efec997a640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeb0003f20 0x7efeb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 -- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 msgr2=0x7efeb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec40cf660 0x7efeb00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 -- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeb0003f20 msgr2=0x7efeb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeb0003f20 0x7efeb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.227+0000 7efeca97c640 1 -- 172.21.15.148:0/4093426933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0003520 con 0x7efeb0004d40 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.228+0000 7efeca97c640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efeb00051c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efebc001910 tx=0x7efeb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd1a5acdc622cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.228+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0014e0 con 0x7efeb0004d40 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.228+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc037070 con 0x7efeb0004d40 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.228+0000 7efec9179640 1 -- 172.21.15.148:0/4093426933 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc032880 con 0x7efeb0004d40 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 -- 172.21.15.148:0/4093426933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 msgr2=0x7efeb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 --2- 172.21.15.148:0/4093426933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efeb00051c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efebc001910 tx=0x7efeb0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efeca97c640 1 -- 172.21.15.148:0/4093426933 reap_dead start 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 -- 172.21.15.148:0/4093426933 shutdown_connections 2024-03-04T12:12:05.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 -- 172.21.15.148:0/4093426933 >> 172.21.15.148:0/4093426933 conn(0x7efec40b33a0 msgr2=0x7efec40b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 -- 172.21.15.148:0/4093426933 shutdown_connections 2024-03-04T12:12:05.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.229+0000 7efecc3df640 1 -- 172.21.15.148:0/4093426933 wait complete. 2024-03-04T12:12:05.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.230+0000 7efecc3df640 1 Processor -- start 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.230+0000 7efecc3df640 1 -- start start 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec41f6ca0 0x7efec415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec41f3e90 0x7efec40c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec415de00 con 0x7efeb0004d40 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec40cc920 con 0x7efec41f6ca0 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40cfac0 con 0x7efec41f3e90 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48234/0 (socket says 172.21.15.148:48234) 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 -- 172.21.15.148:0/1173490540 learned_addr learned my addr 172.21.15.148:0/1173490540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca97c640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec41f3e90 0x7efec40c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efec997a640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec41f6ca0 0x7efec415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 -- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec41f3e90 msgr2=0x7efec40c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec41f3e90 0x7efec40c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 -- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec41f6ca0 msgr2=0x7efec415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec41f6ca0 0x7efec415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 -- 172.21.15.148:0/1173490540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec41f7410 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efeca17b640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efec00079d0 tx=0x7efec000a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0017020 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000ad70 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec41f76f0 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0013610 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.231+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec41f3970 con 0x7efeb0004d40 2024-03-04T12:12:05.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.232+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec41f3b70 con 0x7efeb0004d40 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.233+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efec001a020 con 0x7efeb0004d40 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.233+0000 7efebaffd640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 0x7efea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.233+0000 7efec997a640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 0x7efea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.233+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efec008ae00 con 0x7efeb0004d40 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.234+0000 7efec997a640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 0x7efea806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efec40ca4e0 tx=0x7efeb4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.238+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efec41f3b70 con 0x7efeb0004d40 2024-03-04T12:12:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.408+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efec4067f30 con 0x7efeb0004d40 2024-03-04T12:12:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.409+0000 7efebaffd640 1 -- 172.21.15.148:0/1173490540 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7efec0021020 con 0x7efeb0004d40 2024-03-04T12:12:05.410 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 msgr2=0x7efea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 0x7efea806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efec40ca4e0 tx=0x7efeb4009190 comp rx=0 tx=0).stop 2024-03-04T12:12:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 msgr2=0x7efec40cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efec00079d0 tx=0x7efec000a6c0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 shutdown_connections 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efea806b4d0 0x7efea806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efec41f3e90 0x7efec40c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec41f6ca0 0x7efec415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 --2- 172.21.15.148:0/1173490540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeb0004d40 0x7efec40cacd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 >> 172.21.15.148:0/1173490540 conn(0x7efec40b33a0 msgr2=0x7efec40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 shutdown_connections 2024-03-04T12:12:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.411+0000 7efecc3df640 1 -- 172.21.15.148:0/1173490540 wait complete. 2024-03-04T12:12:05.423 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:05.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.525+0000 7fe98c3ab640 1 Processor -- start 2024-03-04T12:12:05.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.525+0000 7fe98c3ab640 1 -- start start 2024-03-04T12:12:05.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98c3ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe98406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98c3ab640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe9840cfa40 0x7fe9840cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98c3ab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98406bb60 con 0x7fe98415df00 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98c3ab640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9840701d0 con 0x7fe9840cfa40 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98c3ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840704b0 con 0x7fe98406b700 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.526+0000 7fe98a147640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe98415df00 0x7fe9840cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52254/0 (socket says 172.21.15.148:52254) 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a147640 1 -- 172.21.15.148:0/2020068222 learned_addr learned my addr 172.21.15.148:0/2020068222 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> v2:172.21.15.191:3300/0 conn(0x7fe9840cfa40 0x7fe9840cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989946640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe98406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268088651 0 0) 0x7fe98406bb60 con 0x7fe98415df00 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe970003520 con 0x7fe98415df00 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042151539 0 0) 0x7fe970003520 con 0x7fe98415df00 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 msgr2=0x7fe98406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe98406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 >> v2:172.21.15.191:3300/0 conn(0x7fe9840cfa40 msgr2=0x7fe9840cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 --2- 172.21.15.148:0/2020068222 >> v2:172.21.15.191:3300/0 conn(0x7fe9840cfa40 0x7fe9840cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9840cc8e0 con 0x7fe98415df00 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fe97c001670 con 0x7fe98415df00 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 >> v1:172.21.15.148:6789/0 conn(0x7fe98415df00 legacy=0x7fe9840cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9700039b0 0x7fe970003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970004d40 0x7fe9700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840701d0 con 0x7fe970004d40 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840704b0 con 0x7fe9700039b0 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97000b9c0 con 0x7fe98406b700 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> v2:172.21.15.191:3300/0 conn(0x7fe9840cfa40 0x7fe9840cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe989946640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970004d40 0x7fe9700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a147640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9700039b0 0x7fe970003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 -- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 msgr2=0x7fe970004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98406b700 0x7fe970004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 -- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9700039b0 msgr2=0x7fe970003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.527+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9700039b0 0x7fe970003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.528+0000 7fe98a948640 1 -- 172.21.15.148:0/2020068222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970003520 con 0x7fe970004d40 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.528+0000 7fe98a948640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970004d40 0x7fe9700051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e350 tx=0x7fe970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38493d23bf75d571 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.528+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980009dc0 con 0x7fe970004d40 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.528+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98002cde0 con 0x7fe970004d40 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.528+0000 7fe989145640 1 -- 172.21.15.148:0/2020068222 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003b90 con 0x7fe970004d40 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 -- 172.21.15.148:0/2020068222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970004d40 msgr2=0x7fe9700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/2020068222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970004d40 0x7fe9700051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e350 tx=0x7fe970005830 comp rx=0 tx=0).stop 2024-03-04T12:12:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98a948640 1 -- 172.21.15.148:0/2020068222 reap_dead start 2024-03-04T12:12:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 -- 172.21.15.148:0/2020068222 shutdown_connections 2024-03-04T12:12:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 -- 172.21.15.148:0/2020068222 >> 172.21.15.148:0/2020068222 conn(0x7fe9840b2250 msgr2=0x7fe9840b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 -- 172.21.15.148:0/2020068222 shutdown_connections 2024-03-04T12:12:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.529+0000 7fe98c3ab640 1 -- 172.21.15.148:0/2020068222 wait complete. 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.530+0000 7fe98c3ab640 1 Processor -- start 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.530+0000 7fe98c3ab640 1 -- start start 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 0x7fe984150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe984160380 0x7fe984160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840d0410 con 0x7fe98415aaa0 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840ccea0 con 0x7fe984160380 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98c3ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98406fa80 con 0x7fe970004d40 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48256/0 (socket says 172.21.15.148:48256) 2024-03-04T12:12:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 -- 172.21.15.148:0/3116043139 learned_addr learned my addr 172.21.15.148:0/3116043139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe989946640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe984160380 0x7fe984160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a147640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 0x7fe984150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 -- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 msgr2=0x7fe984150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 0x7fe984150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 -- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe984160380 msgr2=0x7fe984160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe984160380 0x7fe984160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 -- 172.21.15.148:0/3116043139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984160e80 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a147640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 0x7fe984150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.531+0000 7fe98a948640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe98002c850 tx=0x7fe980004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.532+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980009dc0 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.532+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980034e20 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.532+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98415f460 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.532+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003990 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.532+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98415fa20 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.533+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98415b460 con 0x7fe98415aaa0 2024-03-04T12:12:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.533+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe980047020 con 0x7fe98415aaa0 2024-03-04T12:12:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.533+0000 7fe97affd640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 0x7fe96806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.534+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9800ac980 con 0x7fe98415aaa0 2024-03-04T12:12:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.534+0000 7fe98a147640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 0x7fe96806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.534+0000 7fe98a147640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 0x7fe96806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.538+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe98007b550 con 0x7fe98415aaa0 2024-03-04T12:12:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.708+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe9840afa30 con 0x7fe98415aaa0 2024-03-04T12:12:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.708+0000 7fe97affd640 1 -- 172.21.15.148:0/3116043139 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe9840afa30 con 0x7fe98415aaa0 2024-03-04T12:12:05.709 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 msgr2=0x7fe96806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 0x7fe96806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 msgr2=0x7fe98415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe98002c850 tx=0x7fe980004ab0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 shutdown_connections 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe96806b4d0 0x7fe96806d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98415aaa0 0x7fe98415af20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe984160380 0x7fe984160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 --2- 172.21.15.148:0/3116043139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe970004d40 0x7fe984150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 >> 172.21.15.148:0/3116043139 conn(0x7fe9840b2250 msgr2=0x7fe9840cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 shutdown_connections 2024-03-04T12:12:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.710+0000 7fe98c3ab640 1 -- 172.21.15.148:0/3116043139 wait complete. 2024-03-04T12:12:05.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:05.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.822+0000 7f6162206640 1 Processor -- start 2024-03-04T12:12:05.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.822+0000 7f6162206640 1 -- start start 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f6162206640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f615c15bb20 0x7f615c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f6162206640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c06d8b0 con 0x7f615c15b030 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f6162206640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c06db90 con 0x7f615c06d3b0 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f6162206640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c0b2720 con 0x7f615c15bb20 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f615b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f615c15b030 0x7f615c15dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52260/0 (socket says 172.21.15.148:52260) 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f615b7fe640 1 -- 172.21.15.148:0/1210504676 learned_addr learned my addr 172.21.15.148:0/1210504676 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f615bfff640 1 --2- 172.21.15.148:0/1210504676 >> v2:172.21.15.191:3300/0 conn(0x7f615c15bb20 0x7f615c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514010867 0 0) 0x7f615c06d8b0 con 0x7f615c15b030 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.823+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6144003520 con 0x7f615c15b030 2024-03-04T12:12:05.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690688462 0 0) 0x7f6144003520 con 0x7f615c15b030 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 >> v2:172.21.15.191:3300/0 conn(0x7f615c15bb20 msgr2=0x7f615c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 --2- 172.21.15.148:0/1210504676 >> v2:172.21.15.191:3300/0 conn(0x7f615c15bb20 0x7f615c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 >> v1:172.21.15.191:6789/0 conn(0x7f615c06d3b0 legacy=0x7f615c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c06de70 con 0x7f615c15b030 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f614c001670 con 0x7f615c15b030 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 >> v1:172.21.15.148:6789/0 conn(0x7f615c15b030 legacy=0x7f615c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61440039b0 0x7f6144003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6144004800 0x7f6144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f614400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0b2720 con 0x7f61440051c0 2024-03-04T12:12:05.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614400c110 con 0x7f61440039b0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614400c310 con 0x7f6144004800 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f614400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615b7fe640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61440039b0 0x7f6144003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615affd640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6144004800 0x7f6144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 -- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6144004800 msgr2=0x7f6144004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6144004800 0x7f6144004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 -- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61440039b0 msgr2=0x7f6144003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61440039b0 0x7f6144003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.824+0000 7f615bfff640 1 -- 172.21.15.148:0/1210504676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144003520 con 0x7f61440051c0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.825+0000 7f615bfff640 1 -- 172.21.15.148:0/1210504676 reap_dead start 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.825+0000 7f615bfff640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f614400b9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6154009fc0 tx=0x7f6144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88358a37c536b1cd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.825+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154009c10 con 0x7f61440051c0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.825+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154003990 con 0x7f61440051c0 2024-03-04T12:12:05.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.825+0000 7f615a7fc640 1 -- 172.21.15.148:0/1210504676 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154003d80 con 0x7f61440051c0 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 -- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 msgr2=0x7f614400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f614400b9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6154009fc0 tx=0x7f6144005830 comp rx=0 tx=0).stop 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 -- 172.21.15.148:0/1210504676 shutdown_connections 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 --2- 172.21.15.148:0/1210504676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f614400b9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 -- 172.21.15.148:0/1210504676 >> 172.21.15.148:0/1210504676 conn(0x7f615c06acd0 msgr2=0x7f615c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 -- 172.21.15.148:0/1210504676 shutdown_connections 2024-03-04T12:12:05.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.826+0000 7f6162206640 1 -- 172.21.15.148:0/1210504676 wait complete. 2024-03-04T12:12:05.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.827+0000 7f6162206640 1 Processor -- start 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.827+0000 7f6162206640 1 -- start start 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f615c14ef30 0x7f615c0717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f615c14e2d0 0x7f615c14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c06e810 con 0x7f61440051c0 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c06e690 con 0x7f615c14e2d0 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c15c120 con 0x7f615c14ef30 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48264/0 (socket says 172.21.15.148:48264) 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 -- 172.21.15.148:0/4234771152 learned_addr learned my addr 172.21.15.148:0/4234771152 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615affd640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f615c14ef30 0x7f615c0717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615bfff640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f615c14e2d0 0x7f615c14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 -- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f615c14ef30 msgr2=0x7f615c0717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f615c14ef30 0x7f615c0717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 -- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f615c14e2d0 msgr2=0x7f615c14e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f615c14e2d0 0x7f615c14e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 -- 172.21.15.148:0/4234771152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c15a880 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f615b7fe640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f614c004f40 tx=0x7f614c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c01a070 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00f7b0 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c14ec90 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00fba0 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.828+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c15a2c0 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.829+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c067680 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.830+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f614c01d070 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.830+0000 7f6143fff640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 0x7f614806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.831+0000 7f615affd640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 0x7f614806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.831+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f614c089c90 con 0x7f61440051c0 2024-03-04T12:12:05.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.831+0000 7f615affd640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 0x7f614806d9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f615c072140 tx=0x7f6150009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:05.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:05.835+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f615c067680 con 0x7f61440051c0 2024-03-04T12:12:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.017+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f615c074f60 con 0x7f61440051c0 2024-03-04T12:12:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.018+0000 7f6143fff640 1 -- 172.21.15.148:0/4234771152 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f614c058860 con 0x7f61440051c0 2024-03-04T12:12:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 msgr2=0x7f614806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 0x7f614806d9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f615c072140 tx=0x7f6150009040 comp rx=0 tx=0).stop 2024-03-04T12:12:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 msgr2=0x7f615c0cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f614c004f40 tx=0x7f614c0074f0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 shutdown_connections 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f614806b500 0x7f614806d9c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f615c14e2d0 0x7f615c14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f615c14ef30 0x7f615c0717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 --2- 172.21.15.148:0/4234771152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61440051c0 0x7f615c0cd830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 >> 172.21.15.148:0/4234771152 conn(0x7f615c06acd0 msgr2=0x7f615c151270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 shutdown_connections 2024-03-04T12:12:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.020+0000 7f6162206640 1 -- 172.21.15.148:0/4234771152 wait complete. 2024-03-04T12:12:06.032 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:06.032 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:06.047 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.186+0000 7f479d0d1640 1 Processor -- start 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.186+0000 7f479d0d1640 1 -- start start 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479d0d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f479815d660 0x7f479815e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479d0d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47980aeb20 0x7f4798071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479d0d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479815da40 con 0x7f4798071890 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479d0d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479815cbc0 con 0x7f47980aeb20 2024-03-04T12:12:06.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479d0d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479815ebb0 con 0x7f479815d660 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47980aeb20 0x7f4798071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479759e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4798071890 0x7f4798150960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52276/0 (socket says 172.21.15.148:52276) 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479759e640 1 -- 172.21.15.148:0/3382098056 learned_addr learned my addr 172.21.15.148:0/3382098056 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f4796d9d640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f479815d660 0x7f479815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479659c640 1 -- 172.21.15.148:0/3382098056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f479815d660 msgr2=0x7f479815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479659c640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f479815d660 0x7f479815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479659c640 1 -- 172.21.15.148:0/3382098056 >> v1:172.21.15.148:6789/0 conn(0x7f4798071890 legacy=0x7f4798150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f4795d9b640 1 -- 172.21.15.148:0/3382098056 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405041537 0 0) 0x7f479815da40 con 0x7f4798071890 2024-03-04T12:12:06.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.187+0000 7f479659c640 1 -- 172.21.15.148:0/3382098056 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980af090 con 0x7f47980aeb20 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.188+0000 7f479659c640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798071350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f478c00a940 tx=0x7f478c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b661009edb5e533a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.188+0000 7f4795d9b640 1 -- 172.21.15.148:0/3382098056 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c03a070 con 0x7f47980aeb20 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.188+0000 7f4795d9b640 1 -- 172.21.15.148:0/3382098056 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c02da20 con 0x7f47980aeb20 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.188+0000 7f4795d9b640 1 -- 172.21.15.148:0/3382098056 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00b040 con 0x7f47980aeb20 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 -- 172.21.15.148:0/3382098056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 msgr2=0x7f4798071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798071350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f478c00a940 tx=0x7f478c02cc70 comp rx=0 tx=0).stop 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 -- 172.21.15.148:0/3382098056 shutdown_connections 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798071350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 --2- 172.21.15.148:0/3382098056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f479815d660 0x7f479815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 -- 172.21.15.148:0/3382098056 >> 172.21.15.148:0/3382098056 conn(0x7f479806acd0 msgr2=0x7f479806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 -- 172.21.15.148:0/3382098056 shutdown_connections 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.189+0000 7f479d0d1640 1 -- 172.21.15.148:0/3382098056 wait complete. 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.190+0000 7f479d0d1640 1 Processor -- start 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.190+0000 7f479d0d1640 1 -- start start 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 0x7f4798143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f479815d660 0x7f47981423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815afb0 con 0x7f47980aeb20 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815acd0 con 0x7f479815d660 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479d0d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479815a660 con 0x7f4798071890 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48288/0 (socket says 172.21.15.148:48288) 2024-03-04T12:12:06.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 -- 172.21.15.148:0/2072312491 learned_addr learned my addr 172.21.15.148:0/2072312491 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f4796d9d640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 0x7f4798143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479759e640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f479815d660 0x7f47981423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 -- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 msgr2=0x7f4798143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 0x7f4798143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 -- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f479815d660 msgr2=0x7f47981423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f479815d660 0x7f47981423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f479659c640 1 -- 172.21.15.148:0/2072312491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798142d80 con 0x7f47980aeb20 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.191+0000 7f4796d9d640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 0x7f4798143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f479659c640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f478c000c00 tx=0x7f478c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c045070 con 0x7f47980aeb20 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c036d10 con 0x7f47980aeb20 2024-03-04T12:12:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4798143060 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c03a070 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.192+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798143620 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.193+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798074f60 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.193+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f478c048070 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.194+0000 7f477f7fe640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 0x7f478406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.194+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f478c0abe20 con 0x7f47980aeb20 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.194+0000 7f4796d9d640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 0x7f478406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.194+0000 7f4796d9d640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 0x7f478406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4788006fd0 tx=0x7f4788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.198+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f478c07a8d0 con 0x7f47980aeb20 2024-03-04T12:12:06.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.368+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4798071350 con 0x7f47980aeb20 2024-03-04T12:12:06.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.369+0000 7f477f7fe640 1 -- 172.21.15.148:0/2072312491 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f478c07a1d0 con 0x7f47980aeb20 2024-03-04T12:12:06.371 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:06.371 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 msgr2=0x7f478406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 0x7f478406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4788006fd0 tx=0x7f4788008040 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 msgr2=0x7f4798141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f478c000c00 tx=0x7f478c0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 shutdown_connections 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f478406b4d0 0x7f478406d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f479815d660 0x7f47981423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47980aeb20 0x7f4798141ae0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 --2- 172.21.15.148:0/2072312491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4798071890 0x7f4798143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 >> 172.21.15.148:0/2072312491 conn(0x7f479806acd0 msgr2=0x7f47980b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 shutdown_connections 2024-03-04T12:12:06.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.371+0000 7f479d0d1640 1 -- 172.21.15.148:0/2072312491 wait complete. 2024-03-04T12:12:06.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:06.384 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:06.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.486+0000 7ff798acd640 1 Processor -- start 2024-03-04T12:12:06.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 -- start start 2024-03-04T12:12:06.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 0x7ff7940b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff7940cbfb0 0x7ff7940b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79406d8a0 con 0x7ff79406d3c0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79406db80 con 0x7ff7940cbfb0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff798acd640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79406de60 con 0x7ff79415a810 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff792d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff7940cbfb0 0x7ff7940b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff792d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff7940cbfb0 0x7ff7940b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48298/0 (socket says 172.21.15.148:48298) 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.487+0000 7ff792d9d640 1 -- 172.21.15.148:0/3012965462 learned_addr learned my addr 172.21.15.148:0/3012965462 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff79259c640 1 --2- 172.21.15.148:0/3012965462 >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 0x7ff7940b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff792d9d640 1 -- 172.21.15.148:0/3012965462 >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 msgr2=0x7ff7940b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff792d9d640 1 --2- 172.21.15.148:0/3012965462 >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 0x7ff7940b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff792d9d640 1 -- 172.21.15.148:0/3012965462 >> v1:172.21.15.191:6789/0 conn(0x7ff79406d3c0 legacy=0x7ff79415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff792d9d640 1 -- 172.21.15.148:0/3012965462 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79406e140 con 0x7ff7940cbfb0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff79259c640 1 --2- 172.21.15.148:0/3012965462 >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 0x7ff7940b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.488+0000 7ff792d9d640 1 --2- 172.21.15.148:0/3012965462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b46d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff79406e560 tx=0x7ff78c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739eac1c9df21494 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff791d9b640 1 -- 172.21.15.148:0/3012965462 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c02ea90 con 0x7ff7940cbfb0 2024-03-04T12:12:06.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff791d9b640 1 -- 172.21.15.148:0/3012965462 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c02ec30 con 0x7ff7940cbfb0 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff791d9b640 1 -- 172.21.15.148:0/3012965462 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c009600 con 0x7ff7940cbfb0 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 -- 172.21.15.148:0/3012965462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7940cbfb0 msgr2=0x7ff7940b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 --2- 172.21.15.148:0/3012965462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b46d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff79406e560 tx=0x7ff78c02e0f0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 -- 172.21.15.148:0/3012965462 shutdown_connections 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 --2- 172.21.15.148:0/3012965462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b46d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 --2- 172.21.15.148:0/3012965462 >> v2:172.21.15.191:3300/0 conn(0x7ff79415a810 0x7ff7940b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 -- 172.21.15.148:0/3012965462 >> 172.21.15.148:0/3012965462 conn(0x7ff79406acd0 msgr2=0x7ff79406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 -- 172.21.15.148:0/3012965462 shutdown_connections 2024-03-04T12:12:06.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.489+0000 7ff798acd640 1 -- 172.21.15.148:0/3012965462 wait complete. 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.490+0000 7ff798acd640 1 Processor -- start 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.490+0000 7ff798acd640 1 -- start start 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff79406d3c0 0x7ff7940cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415e3c0 con 0x7ff79415a810 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b3c50 con 0x7ff7940cbfb0 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff798acd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415e540 con 0x7ff79406d3c0 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48308/0 (socket says 172.21.15.148:48308) 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff792d9d640 1 -- 172.21.15.148:0/2932085262 learned_addr learned my addr 172.21.15.148:0/2932085262 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff79259c640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff79406d3c0 0x7ff7940cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.491+0000 7ff78bfff640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 -- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff79406d3c0 msgr2=0x7ff7940cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff79406d3c0 0x7ff7940cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 -- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7940cbfb0 msgr2=0x7ff7940b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 -- 172.21.15.148:0/2932085262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940cf310 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff792d9d640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff78c02e700 tx=0x7ff78c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c045020 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c040660 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7940cdb80 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00b720 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.492+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940ce140 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.493+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794074f60 con 0x7ff79415a810 2024-03-04T12:12:06.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.494+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff78c048020 con 0x7ff79415a810 2024-03-04T12:12:06.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.494+0000 7ff78affd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.494+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff78c0aeeb0 con 0x7ff79415a810 2024-03-04T12:12:06.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.494+0000 7ff79259c640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.495+0000 7ff79259c640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 0x7ff77406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7940cee10 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.498+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff78c04f030 con 0x7ff79415a810 2024-03-04T12:12:06.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.668+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff79406ec30 con 0x7ff79415a810 2024-03-04T12:12:06.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.668+0000 7ff78affd640 1 -- 172.21.15.148:0/2932085262 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff78c07d3b0 con 0x7ff79415a810 2024-03-04T12:12:06.669 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 msgr2=0x7ff77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 0x7ff77406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7940cee10 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 msgr2=0x7ff79414edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff78c02e700 tx=0x7ff78c004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 shutdown_connections 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79415a810 0x7ff79414edf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7940cbfb0 0x7ff7940b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 --2- 172.21.15.148:0/2932085262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff79406d3c0 0x7ff7940cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 >> 172.21.15.148:0/2932085262 conn(0x7ff79406acd0 msgr2=0x7ff7940afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 shutdown_connections 2024-03-04T12:12:06.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.670+0000 7ff798acd640 1 -- 172.21.15.148:0/2932085262 wait complete. 2024-03-04T12:12:06.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:06.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.785+0000 7fa48c081640 1 Processor -- start 2024-03-04T12:12:06.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.785+0000 7fa48c081640 1 -- start start 2024-03-04T12:12:06.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48c081640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 0x7fa4840b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48c081640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840ca970 con 0x7fa48415de40 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48c081640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa484161d90 con 0x7fa4840cec40 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48c081640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa484162070 con 0x7fa4840cf020 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48961c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa48415de40 0x7fa4840d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52286/0 (socket says 172.21.15.148:52286) 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48961c640 1 -- 172.21.15.148:0/3501198783 learned_addr learned my addr 172.21.15.148:0/3501198783 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48a61e640 1 --2- 172.21.15.148:0/3501198783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 0x7fa4840b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820549948 0 0) 0x7fa4840ca970 con 0x7fa48415de40 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa470003520 con 0x7fa48415de40 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306015811 0 0) 0x7fa484161d90 con 0x7fa4840cec40 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48a61e640 1 -- 172.21.15.148:0/3501198783 >> v1:172.21.15.191:6789/0 conn(0x7fa4840cec40 legacy=0x7fa48415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48a61e640 1 -- 172.21.15.148:0/3501198783 >> v1:172.21.15.148:6789/0 conn(0x7fa48415de40 legacy=0x7fa4840d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.786+0000 7fa48a61e640 1 -- 172.21.15.148:0/3501198783 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484162350 con 0x7fa4840cf020 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.787+0000 7fa48a61e640 1 --2- 172.21.15.148:0/3501198783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 0x7fa4840b6ea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009dc0 tx=0x7fa47c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f287b0898a93b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.787+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c039070 con 0x7fa4840cf020 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.787+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c02ccd0 con 0x7fa4840cf020 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.787+0000 7fa488e1b640 1 -- 172.21.15.148:0/3501198783 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c034640 con 0x7fa4840cf020 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 -- 172.21.15.148:0/3501198783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 msgr2=0x7fa4840b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 --2- 172.21.15.148:0/3501198783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 0x7fa4840b6ea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009dc0 tx=0x7fa47c02bf20 comp rx=0 tx=0).stop 2024-03-04T12:12:06.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 -- 172.21.15.148:0/3501198783 shutdown_connections 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 --2- 172.21.15.148:0/3501198783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cf020 0x7fa4840b6ea0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 -- 172.21.15.148:0/3501198783 >> 172.21.15.148:0/3501198783 conn(0x7fa4840d0580 msgr2=0x7fa4840d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 -- 172.21.15.148:0/3501198783 shutdown_connections 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.788+0000 7fa48c081640 1 -- 172.21.15.148:0/3501198783 wait complete. 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.789+0000 7fa48c081640 1 Processor -- start 2024-03-04T12:12:06.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.789+0000 7fa48c081640 1 -- start start 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cec40 0x7fa48415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa48415de40 0x7fa4840cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840d3300 con 0x7fa4840cf020 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840d1bf0 con 0x7fa48415de40 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48c081640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840d3480 con 0x7fa4840cec40 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48316/0 (socket says 172.21.15.148:48316) 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 -- 172.21.15.148:0/2498583033 learned_addr learned my addr 172.21.15.148:0/2498583033 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48a61e640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa48415de40 0x7fa4840cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa489e1d640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cec40 0x7fa48415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 -- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cec40 msgr2=0x7fa48415ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cec40 0x7fa48415ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 -- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa48415de40 msgr2=0x7fa4840cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa48415de40 0x7fa4840cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 -- 172.21.15.148:0/2498583033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840b5b30 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.790+0000 7fa48961c640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4740010e0 tx=0x7fa47400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.791+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474017070 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.791+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa474005d00 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.791+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48415f2b0 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.791+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740072b0 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.791+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4840c78f0 con 0x7fa4840cf020 2024-03-04T12:12:06.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.792+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa484074f60 con 0x7fa4840cf020 2024-03-04T12:12:06.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.792+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa47401a070 con 0x7fa4840cf020 2024-03-04T12:12:06.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.793+0000 7fa47a7fc640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 0x7fa46806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:06.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.793+0000 7fa489e1d640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 0x7fa46806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:06.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.793+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa4740898d0 con 0x7fa4840cf020 2024-03-04T12:12:06.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.793+0000 7fa489e1d640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 0x7fa46806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa480000f60 tx=0x7fa480009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:06.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.797+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa474057dd0 con 0x7fa4840cf020 2024-03-04T12:12:06.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.973+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4840d1930 con 0x7fa4840cf020 2024-03-04T12:12:06.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.973+0000 7fa47a7fc640 1 -- 172.21.15.148:0/2498583033 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa47405be70 con 0x7fa4840cf020 2024-03-04T12:12:06.975 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 msgr2=0x7fa46806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 0x7fa46806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa480000f60 tx=0x7fa480009040 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 msgr2=0x7fa4840c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4740010e0 tx=0x7fa47400c960 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 shutdown_connections 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa46806b4d0 0x7fa46806d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa48415de40 0x7fa4840cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4840cf020 0x7fa4840c9ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 --2- 172.21.15.148:0/2498583033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4840cec40 0x7fa48415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 >> 172.21.15.148:0/2498583033 conn(0x7fa4840d0580 msgr2=0x7fa484159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:06.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 shutdown_connections 2024-03-04T12:12:06.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:06.975+0000 7fa48c081640 1 -- 172.21.15.148:0/2498583033 wait complete. 2024-03-04T12:12:06.987 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:07.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.091+0000 7f3f1cd7e640 1 Processor -- start 2024-03-04T12:12:07.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.091+0000 7f3f1cd7e640 1 -- start start 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1cd7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1cd7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f180ceec0 0x7f3f180b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1cd7e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1815ca50 con 0x7f3f1815c2e0 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1cd7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1815d2f0 con 0x7f3f180ceec0 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1cd7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815d6d0 con 0x7f3f180ccbd0 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f180ceec0 0x7f3f180b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f180ceec0 0x7f3f180b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48326/0 (socket says 172.21.15.148:48326) 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 -- 172.21.15.148:0/3742093057 learned_addr learned my addr 172.21.15.148:0/3742093057 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1659c640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 -- 172.21.15.148:0/3742093057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 msgr2=0x7f3f180b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 -- 172.21.15.148:0/3742093057 >> v1:172.21.15.191:6789/0 conn(0x7f3f1815c2e0 legacy=0x7f3f180ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f16d9d640 1 -- 172.21.15.148:0/3742093057 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1815cd30 con 0x7f3f180ceec0 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.092+0000 7f3f1659c640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.093+0000 7f3f16d9d640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180b0560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009e50 tx=0x7f3f1002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103085a4826a214f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.093+0000 7f3f15d9b640 1 -- 172.21.15.148:0/3742093057 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1002cb00 con 0x7f3f180ceec0 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.093+0000 7f3f15d9b640 1 -- 172.21.15.148:0/3742093057 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1002cca0 con 0x7f3f180ceec0 2024-03-04T12:12:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.093+0000 7f3f15d9b640 1 -- 172.21.15.148:0/3742093057 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10034610 con 0x7f3f180ceec0 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/3742093057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 msgr2=0x7f3f180b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180b0560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009e50 tx=0x7f3f1002bf60 comp rx=0 tx=0).stop 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/3742093057 shutdown_connections 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180b0560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/3742093057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/3742093057 >> 172.21.15.148:0/3742093057 conn(0x7f3f18066dc0 msgr2=0x7f3f180671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/3742093057 shutdown_connections 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.094+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/3742093057 wait complete. 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.095+0000 7f3f1cd7e640 1 Processor -- start 2024-03-04T12:12:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.095+0000 7f3f1cd7e640 1 -- start start 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1815c2e0 0x7f3f181f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180ccfb0 con 0x7f3f180ceec0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815c760 con 0x7f3f1815c2e0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180b0aa0 con 0x7f3f180ccbd0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48338/0 (socket says 172.21.15.148:48338) 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 -- 172.21.15.148:0/4286077902 learned_addr learned my addr 172.21.15.148:0/4286077902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1659c640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f16d9d640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1815c2e0 0x7f3f181f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 -- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 msgr2=0x7f3f180c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 -- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1815c2e0 msgr2=0x7f3f181f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1815c2e0 0x7f3f181f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 -- 172.21.15.148:0/4286077902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180c6eb0 con 0x7f3f180ceec0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0ffff640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c3c0 tx=0x7f3f0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f000052d0 con 0x7f3f180ceec0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f00005490 con 0x7f3f180ceec0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f180c7190 con 0x7f3f180ceec0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00007260 con 0x7f3f180ceec0 2024-03-04T12:12:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.096+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f180c7750 con 0x7f3f180ceec0 2024-03-04T12:12:07.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.098+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f180c7920 con 0x7f3f180ceec0 2024-03-04T12:12:07.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.098+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3f00007400 con 0x7f3f180ceec0 2024-03-04T12:12:07.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.098+0000 7f3f0effd640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.098+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f00089920 con 0x7f3f180ceec0 2024-03-04T12:12:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.098+0000 7f3f1659c640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.099+0000 7f3f1659c640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 0x7f3ef806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f180b37f0 tx=0x7f3f08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.102+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f00057e10 con 0x7f3f180ceec0 2024-03-04T12:12:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.274+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f18074f60 con 0x7f3f180ceec0 2024-03-04T12:12:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.275+0000 7f3f0effd640 1 -- 172.21.15.148:0/4286077902 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f0005beb0 con 0x7f3f180ceec0 2024-03-04T12:12:07.277 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 msgr2=0x7f3ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 0x7f3ef806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f180b37f0 tx=0x7f3f08008040 comp rx=0 tx=0).stop 2024-03-04T12:12:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 msgr2=0x7f3f180c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c3c0 tx=0x7f3f0000c880 comp rx=0 tx=0).stop 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 shutdown_connections 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1815c2e0 0x7f3f181f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f180ceec0 0x7f3f180c6680 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 --2- 172.21.15.148:0/4286077902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f180ccbd0 0x7f3f180c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 >> 172.21.15.148:0/4286077902 conn(0x7f3f18066dc0 msgr2=0x7f3f180b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 shutdown_connections 2024-03-04T12:12:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.277+0000 7f3f1cd7e640 1 -- 172.21.15.148:0/4286077902 wait complete. 2024-03-04T12:12:07.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:07.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.392+0000 7fb414f37640 1 Processor -- start 2024-03-04T12:12:07.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.392+0000 7fb414f37640 1 -- start start 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb414f37640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb41015f660 0x7fb4100d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb414f37640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4100d1cc0 0x7fb4100cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb414f37640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41015bf50 con 0x7fb41015bb70 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb414f37640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100cf890 con 0x7fb4100d1cc0 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb414f37640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100b1150 con 0x7fb41015f660 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4100d1cc0 0x7fb4100cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb41015bb70 0x7fb4100b0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52288/0 (socket says 172.21.15.148:52288) 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40e59c640 1 -- 172.21.15.148:0/1520182236 learned_addr learned my addr 172.21.15.148:0/1520182236 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:07.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb407fff640 1 --2- 172.21.15.148:0/1520182236 >> v2:172.21.15.191:3300/0 conn(0x7fb41015f660 0x7fb4100d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40ed9d640 1 -- 172.21.15.148:0/1520182236 >> v2:172.21.15.191:3300/0 conn(0x7fb41015f660 msgr2=0x7fb4100d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40ed9d640 1 --2- 172.21.15.148:0/1520182236 >> v2:172.21.15.191:3300/0 conn(0x7fb41015f660 0x7fb4100d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40ed9d640 1 -- 172.21.15.148:0/1520182236 >> v1:172.21.15.148:6789/0 conn(0x7fb41015bb70 legacy=0x7fb4100b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40ed9d640 1 -- 172.21.15.148:0/1520182236 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41015fb60 con 0x7fb4100d1cc0 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.393+0000 7fb40dd9b640 1 -- 172.21.15.148:0/1520182236 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894515214 0 0) 0x7fb41015bf50 con 0x7fb41015bb70 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb40ed9d640 1 --2- 172.21.15.148:0/1520182236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100cfd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4100705e0 tx=0x7fb40802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a74ee9a1be77bf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb40dd9b640 1 -- 172.21.15.148:0/1520182236 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408039070 con 0x7fb4100d1cc0 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb40dd9b640 1 -- 172.21.15.148:0/1520182236 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40802cbf0 con 0x7fb4100d1cc0 2024-03-04T12:12:07.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb40dd9b640 1 -- 172.21.15.148:0/1520182236 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408034440 con 0x7fb4100d1cc0 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 -- 172.21.15.148:0/1520182236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 msgr2=0x7fb4100cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 --2- 172.21.15.148:0/1520182236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100cfd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4100705e0 tx=0x7fb40802be40 comp rx=0 tx=0).stop 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 -- 172.21.15.148:0/1520182236 shutdown_connections 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 --2- 172.21.15.148:0/1520182236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100cfd20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 --2- 172.21.15.148:0/1520182236 >> v2:172.21.15.191:3300/0 conn(0x7fb41015f660 0x7fb4100d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 -- 172.21.15.148:0/1520182236 >> 172.21.15.148:0/1520182236 conn(0x7fb4100b33a0 msgr2=0x7fb4100b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 -- 172.21.15.148:0/1520182236 shutdown_connections 2024-03-04T12:12:07.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.394+0000 7fb414f37640 1 -- 172.21.15.148:0/1520182236 wait complete. 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 Processor -- start 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 -- start start 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41015bb70 0x7fb4100b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb41015f660 0x7fb4100a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410160400 con 0x7fb4100d1cc0 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410160120 con 0x7fb41015bb70 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.396+0000 7fb414f37640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100d0260 con 0x7fb41015f660 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48352/0 (socket says 172.21.15.148:48352) 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 -- 172.21.15.148:0/1394729501 learned_addr learned my addr 172.21.15.148:0/1394729501 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb407fff640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41015bb70 0x7fb4100b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 -- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb41015f660 msgr2=0x7fb4100a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40ed9d640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb41015f660 0x7fb4100a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb41015f660 0x7fb4100a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 -- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41015bb70 msgr2=0x7fb4100b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41015bb70 0x7fb4100b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 -- 172.21.15.148:0/1394729501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100a8d70 con 0x7fb4100d1cc0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb40e59c640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c500 tx=0x7fb40000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400018070 con 0x7fb4100d1cc0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400007580 con 0x7fb4100d1cc0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400004a20 con 0x7fb4100d1cc0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4100a9050 con 0x7fb4100d1cc0 2024-03-04T12:12:07.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.397+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4100a9610 con 0x7fb4100d1cc0 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.398+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410074f60 con 0x7fb4100d1cc0 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.399+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb40001b070 con 0x7fb4100d1cc0 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.399+0000 7fb406ffd640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.399+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb4000897e0 con 0x7fb4100d1cc0 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.399+0000 7fb407fff640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.400+0000 7fb407fff640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 0x7fb3f006d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8006fd0 tx=0x7fb3f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.403+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4000583b0 con 0x7fb4100d1cc0 2024-03-04T12:12:07.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.575+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb4100cbba0 con 0x7fb4100d1cc0 2024-03-04T12:12:07.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.575+0000 7fb406ffd640 1 -- 172.21.15.148:0/1394729501 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb400057cb0 con 0x7fb4100d1cc0 2024-03-04T12:12:07.577 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 msgr2=0x7fb3f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 0x7fb3f006d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8006fd0 tx=0x7fb3f8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 msgr2=0x7fb4100a8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c500 tx=0x7fb40000c9c0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 shutdown_connections 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb41015f660 0x7fb4100a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41015bb70 0x7fb4100b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 --2- 172.21.15.148:0/1394729501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4100d1cc0 0x7fb4100a8470 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 >> 172.21.15.148:0/1394729501 conn(0x7fb4100b33a0 msgr2=0x7fb4100b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 shutdown_connections 2024-03-04T12:12:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.577+0000 7fb414f37640 1 -- 172.21.15.148:0/1394729501 wait complete. 2024-03-04T12:12:07.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:07.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.692+0000 7f63f9a75640 1 Processor -- start 2024-03-04T12:12:07.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.692+0000 7f63f9a75640 1 -- start start 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.692+0000 7f63f9a75640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63f41624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f9a75640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f40cf730 con 0x7f63f40d0050 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f9a75640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f40cfcb0 con 0x7f63f40ce360 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f9a75640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40cf9d0 con 0x7f63f415de20 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63ebfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f63f40d0050 0x7f63f415ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52294/0 (socket says 172.21.15.148:52294) 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63ebfff640 1 -- 172.21.15.148:0/805800156 learned_addr learned my addr 172.21.15.148:0/805800156 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610409293 0 0) 0x7f63f40cf730 con 0x7f63f40d0050 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f37fe640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63f41624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63e0003520 con 0x7f63f40d0050 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070662904 0 0) 0x7f63e0003520 con 0x7f63f40d0050 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 msgr2=0x7f63f41624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63f41624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 >> v1:172.21.15.191:6789/0 conn(0x7f63f40ce360 legacy=0x7f63f406c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4161a00 con 0x7f63f40d0050 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f63dc001670 con 0x7f63f40d0050 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 >> v1:172.21.15.148:6789/0 conn(0x7f63f40d0050 legacy=0x7f63f415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e00039e0 0x7f63e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e0004800 0x7f63e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40cf9d0 con 0x7f63e00039e0 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e000ba70 con 0x7f63e0004800 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e000bc70 con 0x7f63f415de20 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f37fe640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e00039e0 0x7f63e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63ebfff640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e0004800 0x7f63e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.693+0000 7f63f3fff640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 -- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 msgr2=0x7f63e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f415de20 0x7f63e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 -- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e0004800 msgr2=0x7f63e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e0004800 0x7f63e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 -- 172.21.15.148:0/805800156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0003520 con 0x7f63e00039e0 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.694+0000 7f63f37fe640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e00039e0 0x7f63e0003e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a90 tx=0x7f63e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a766e5e08b630d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4039070 con 0x7f63e00039e0 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e402c680 con 0x7f63e00039e0 2024-03-04T12:12:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f2ffd640 1 -- 172.21.15.148:0/805800156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4009690 con 0x7f63e00039e0 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 -- 172.21.15.148:0/805800156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e00039e0 msgr2=0x7f63e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 --2- 172.21.15.148:0/805800156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e00039e0 0x7f63e0003e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a90 tx=0x7f63e0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f3fff640 1 -- 172.21.15.148:0/805800156 reap_dead start 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 -- 172.21.15.148:0/805800156 shutdown_connections 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 -- 172.21.15.148:0/805800156 >> 172.21.15.148:0/805800156 conn(0x7f63f406f6e0 msgr2=0x7f63f406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 -- 172.21.15.148:0/805800156 shutdown_connections 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.695+0000 7f63f9a75640 1 -- 172.21.15.148:0/805800156 wait complete. 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.696+0000 7f63f9a75640 1 Processor -- start 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.696+0000 7f63f9a75640 1 -- start start 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e00039e0 0x7f63f41519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f4160f00 0x7f63f415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f41624e0 con 0x7f63f4160a80 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4161fc0 con 0x7f63e00039e0 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63f9a75640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4162660 con 0x7f63f4160f00 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48366/0 (socket says 172.21.15.148:48366) 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 -- 172.21.15.148:0/4220999606 learned_addr learned my addr 172.21.15.148:0/4220999606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 -- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f4160f00 msgr2=0x7f63f415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f4160f00 0x7f63f415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 -- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e00039e0 msgr2=0x7f63f41519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e00039e0 0x7f63f41519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.697+0000 7f63ebfff640 1 -- 172.21.15.148:0/4220999606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40b4be0 con 0x7f63f4160a80 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63ebfff640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63dc000f60 tx=0x7f63dc007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00f8e0 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc018070 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f40b4ec0 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00fca0 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.698+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f40b5480 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.699+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f4074f60 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.699+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f63dc01d460 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.700+0000 7f63f0ff9640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 0x7f63d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.700+0000 7f63f37fe640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 0x7f63d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.700+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f63dc08a650 con 0x7f63f4160a80 2024-03-04T12:12:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.700+0000 7f63f37fe640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 0x7f63d406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63e4036000 tx=0x7f63e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.704+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63dc059190 con 0x7f63f4160a80 2024-03-04T12:12:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.889+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63f40b78d0 con 0x7f63f4160a80 2024-03-04T12:12:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.890+0000 7f63f0ff9640 1 -- 172.21.15.148:0/4220999606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f63dc058a90 con 0x7f63f4160a80 2024-03-04T12:12:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:07.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 msgr2=0x7f63d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 0x7f63d406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63e4036000 tx=0x7f63e402e000 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 msgr2=0x7f63f415cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63dc000f60 tx=0x7f63dc007720 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 shutdown_connections 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f63d406b4d0 0x7f63d406d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63f4160f00 0x7f63f415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63f4160a80 0x7f63f415cc80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 --2- 172.21.15.148:0/4220999606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e00039e0 0x7f63f41519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 >> 172.21.15.148:0/4220999606 conn(0x7f63f406f6e0 msgr2=0x7f63f40c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 shutdown_connections 2024-03-04T12:12:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:07.892+0000 7f63f9a75640 1 -- 172.21.15.148:0/4220999606 wait complete. 2024-03-04T12:12:07.904 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-03-04T12:12:07.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:07.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-04T12:12:07.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[45650]: fuse finished with error 0 and tester_r 0 2024-03-04T12:12:07.931 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T12:12:14.033 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:12:14.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:14.033 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-04T12:12:14.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:14.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.349+0000 7fd807dc6640 1 Processor -- start 2024-03-04T12:12:14.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.349+0000 7fd807dc6640 1 -- start start 2024-03-04T12:12:14.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd807dc6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd80006c470 0x7fd80006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd807dc6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800066de0 0x7fd80015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd807dc6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8000ad6c0 con 0x7fd80006c090 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd807dc6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000ad960 con 0x7fd80006c470 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd807dc6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80006ebc0 con 0x7fd800066de0 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805361640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd80006c470 0x7fd80006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805b62640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd80006c090 0x7fd8000af220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47292/0 (socket says 172.21.15.148:47292) 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805b62640 1 -- 172.21.15.148:0/2087640909 learned_addr learned my addr 172.21.15.148:0/2087640909 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd806363640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800066de0 0x7fd80015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805361640 1 -- 172.21.15.148:0/2087640909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800066de0 msgr2=0x7fd80015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805361640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800066de0 0x7fd80015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805361640 1 -- 172.21.15.148:0/2087640909 >> v1:172.21.15.148:6789/0 conn(0x7fd80006c090 legacy=0x7fd8000af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd805361640 1 -- 172.21.15.148:0/2087640909 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80006eea0 con 0x7fd80006c470 2024-03-04T12:12:14.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.350+0000 7fd804b60640 1 -- 172.21.15.148:0/2087640909 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684058112 0 0) 0x7fd8000ad6c0 con 0x7fd80006c090 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.351+0000 7fd805361640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd80006c470 0x7fd80006f680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00097a0 tx=0x7fd7f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0cc340af9d2dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.351+0000 7fd804b60640 1 -- 172.21.15.148:0/2087640909 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00315a0 con 0x7fd80006c470 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.351+0000 7fd804b60640 1 -- 172.21.15.148:0/2087640909 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f0037070 con 0x7fd80006c470 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.351+0000 7fd804b60640 1 -- 172.21.15.148:0/2087640909 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0031960 con 0x7fd80006c470 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 -- 172.21.15.148:0/2087640909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd80006c470 msgr2=0x7fd80006f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd80006c470 0x7fd80006f680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00097a0 tx=0x7fd7f002b840 comp rx=0 tx=0).stop 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 -- 172.21.15.148:0/2087640909 shutdown_connections 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800066de0 0x7fd80015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 --2- 172.21.15.148:0/2087640909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd80006c470 0x7fd80006f680 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 -- 172.21.15.148:0/2087640909 >> 172.21.15.148:0/2087640909 conn(0x7fd8000cfef0 msgr2=0x7fd8000d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:14.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 -- 172.21.15.148:0/2087640909 shutdown_connections 2024-03-04T12:12:14.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.352+0000 7fd807dc6640 1 -- 172.21.15.148:0/2087640909 wait complete. 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.353+0000 7fd807dc6640 1 Processor -- start 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.353+0000 7fd807dc6640 1 -- start start 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd80006c090 0x7fd8000b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 0x7fd8000b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000addc0 con 0x7fd800066de0 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000adc40 con 0x7fd80006c090 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd807dc6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80015faa0 con 0x7fd800099a00 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39652/0 (socket says 172.21.15.148:39652) 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 -- 172.21.15.148:0/726702482 learned_addr learned my addr 172.21.15.148:0/726702482 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd806363640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 0x7fd8000b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805361640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd80006c090 0x7fd8000b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 -- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 msgr2=0x7fd8000b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 0x7fd8000b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 -- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd80006c090 msgr2=0x7fd8000b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd80006c090 0x7fd8000b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 -- 172.21.15.148:0/726702482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80009d070 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd806363640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 0x7fd8000b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.354+0000 7fd805b62640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c4d0 tx=0x7fd7fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.355+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc017070 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.355+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc005d00 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.355+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80009d350 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.355+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0072b0 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.355+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80009d910 con 0x7fd800066de0 2024-03-04T12:12:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.356+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8000cf200 con 0x7fd800066de0 2024-03-04T12:12:14.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.356+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd7fc01a070 con 0x7fd800066de0 2024-03-04T12:12:14.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.357+0000 7fd7f67fc640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.357+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd7fc0898f0 con 0x7fd800066de0 2024-03-04T12:12:14.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.357+0000 7fd805361640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.357+0000 7fd805361640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 0x7fd7e406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0036000 tx=0x7fd7f003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.361+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8000cf200 con 0x7fd800066de0 2024-03-04T12:12:14.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.536+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd800074f60 con 0x7fd800066de0 2024-03-04T12:12:14.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.537+0000 7fd7f67fc640 1 -- 172.21.15.148:0/726702482 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7fd7fc0584c0 con 0x7fd800066de0 2024-03-04T12:12:14.539 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:14.540 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.539+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 msgr2=0x7fd7e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.539+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 0x7fd7e406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0036000 tx=0x7fd7f003d040 comp rx=0 tx=0).stop 2024-03-04T12:12:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.539+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 msgr2=0x7fd800156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.539+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c4d0 tx=0x7fd7fc00c990 comp rx=0 tx=0).stop 2024-03-04T12:12:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 shutdown_connections 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd800099a00 0x7fd8000b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd80006c090 0x7fd8000b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 --2- 172.21.15.148:0/726702482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800066de0 0x7fd800156d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 >> 172.21.15.148:0/726702482 conn(0x7fd8000cfef0 msgr2=0x7fd80015de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 shutdown_connections 2024-03-04T12:12:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.540+0000 7fd807dc6640 1 -- 172.21.15.148:0/726702482 wait complete. 2024-03-04T12:12:14.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.651+0000 7fad5f0a7640 1 Processor -- start 2024-03-04T12:12:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.651+0000 7fad5f0a7640 1 -- start start 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad5815bbb0 0x7fad580d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad580d0e40 0x7fad580cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad580cc4c0 0x7fad580d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580d2880 con 0x7fad580cc4c0 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580cfb40 con 0x7fad580d0e40 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5f0a7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580cfe20 con 0x7fad5815bbb0 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5d644640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad580cc4c0 0x7fad580d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5d644640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad580cc4c0 0x7fad580d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39658/0 (socket says 172.21.15.148:39658) 2024-03-04T12:12:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.652+0000 7fad5d644640 1 -- 172.21.15.148:0/2291379280 learned_addr learned my addr 172.21.15.148:0/2291379280 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad57fff640 1 --2- 172.21.15.148:0/2291379280 >> v2:172.21.15.191:3300/0 conn(0x7fad580d0e40 0x7fad580cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5ce43640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad5815bbb0 0x7fad580d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 -- 172.21.15.148:0/2291379280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad5815bbb0 msgr2=0x7fad580d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad5815bbb0 0x7fad580d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 -- 172.21.15.148:0/2291379280 >> v2:172.21.15.191:3300/0 conn(0x7fad580d0e40 msgr2=0x7fad580cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 --2- 172.21.15.148:0/2291379280 >> v2:172.21.15.191:3300/0 conn(0x7fad580d0e40 0x7fad580cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 -- 172.21.15.148:0/2291379280 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580d0100 con 0x7fad580cc4c0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.653+0000 7fad5d644640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad580cc4c0 0x7fad580d1ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fad580d2a30 tx=0x7fad5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f639e16ecd739d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad577fe640 1 -- 172.21.15.148:0/2291379280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5002ca70 con 0x7fad580cc4c0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad577fe640 1 -- 172.21.15.148:0/2291379280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5002cc10 con 0x7fad580cc4c0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad577fe640 1 -- 172.21.15.148:0/2291379280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500343f0 con 0x7fad580cc4c0 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 -- 172.21.15.148:0/2291379280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad580cc4c0 msgr2=0x7fad580d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad580cc4c0 0x7fad580d1ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fad580d2a30 tx=0x7fad5002bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 -- 172.21.15.148:0/2291379280 shutdown_connections 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad580cc4c0 0x7fad580d1ca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/2291379280 >> v2:172.21.15.191:3300/0 conn(0x7fad580d0e40 0x7fad580cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/2291379280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad5815bbb0 0x7fad580d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 -- 172.21.15.148:0/2291379280 >> 172.21.15.148:0/2291379280 conn(0x7fad580b3480 msgr2=0x7fad580b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 -- 172.21.15.148:0/2291379280 shutdown_connections 2024-03-04T12:12:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.654+0000 7fad5f0a7640 1 -- 172.21.15.148:0/2291379280 wait complete. 2024-03-04T12:12:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.655+0000 7fad5f0a7640 1 Processor -- start 2024-03-04T12:12:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.655+0000 7fad5f0a7640 1 -- start start 2024-03-04T12:12:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad580cc4c0 0x7fad581e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 0x7fad581e4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5815bf90 con 0x7fad5815bbb0 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580d21e0 con 0x7fad580cc4c0 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5f0a7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580c8b30 con 0x7fad580d0e40 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5d644640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5d644640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39662/0 (socket says 172.21.15.148:39662) 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5d644640 1 -- 172.21.15.148:0/146371175 learned_addr learned my addr 172.21.15.148:0/146371175 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad5ce43640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad580cc4c0 0x7fad581e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.656+0000 7fad57fff640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 0x7fad581e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 -- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 msgr2=0x7fad581e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 0x7fad581e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 -- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad580cc4c0 msgr2=0x7fad581e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad580cc4c0 0x7fad581e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 -- 172.21.15.148:0/146371175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580c8e10 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad57fff640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 0x7fad581e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5d644640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad50036040 tx=0x7fad50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50043020 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5003e450 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad580c90f0 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50009510 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.657+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad580c96b0 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.658+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58074f60 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.658+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fad50046020 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.659+0000 7fad557fa640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.659+0000 7fad5ce43640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.659+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fad500acc60 con 0x7fad5815bbb0 2024-03-04T12:12:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.659+0000 7fad5ce43640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad48007920 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.663+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad5004d030 con 0x7fad5815bbb0 2024-03-04T12:12:14.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.834+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad580cec90 con 0x7fad5815bbb0 2024-03-04T12:12:14.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.836+0000 7fad557fa640 1 -- 172.21.15.148:0/146371175 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7fad5007b830 con 0x7fad5815bbb0 2024-03-04T12:12:14.837 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:14.838 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:14.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 msgr2=0x7fad4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad48007920 tx=0x7fad48008040 comp rx=0 tx=0).stop 2024-03-04T12:12:14.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 msgr2=0x7fad581e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad50036040 tx=0x7fad50004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 shutdown_connections 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad5815bbb0 0x7fad581e4ec0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad580d0e40 0x7fad581e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 --2- 172.21.15.148:0/146371175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad580cc4c0 0x7fad581e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 >> 172.21.15.148:0/146371175 conn(0x7fad580b3480 msgr2=0x7fad580b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 shutdown_connections 2024-03-04T12:12:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.838+0000 7fad5f0a7640 1 -- 172.21.15.148:0/146371175 wait complete. 2024-03-04T12:12:14.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-04T12:12:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.957+0000 7f3d9bafe640 1 Processor -- start 2024-03-04T12:12:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.957+0000 7f3d9bafe640 1 -- start start 2024-03-04T12:12:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9bafe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d940b5700 0x7f3d940d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9bafe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d940d34a0 0x7f3d9406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9bafe640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940d2400 con 0x7f3d940b9410 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9bafe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940d26e0 con 0x7f3d940d34a0 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9bafe640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940d29c0 con 0x7f3d940b5700 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d940d34a0 0x7f3d9406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d940d34a0 0x7f3d9406c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39672/0 (socket says 172.21.15.148:39672) 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 -- 172.21.15.148:0/1543589744 learned_addr learned my addr 172.21.15.148:0/1543589744 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d9989a640 1 --2- 172.21.15.148:0/1543589744 >> v2:172.21.15.191:3300/0 conn(0x7f3d940b5700 0x7f3d940d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 -- 172.21.15.148:0/1543589744 >> v2:172.21.15.191:3300/0 conn(0x7f3d940b5700 msgr2=0x7f3d940d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 --2- 172.21.15.148:0/1543589744 >> v2:172.21.15.191:3300/0 conn(0x7f3d940b5700 0x7f3d940d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 -- 172.21.15.148:0/1543589744 >> v1:172.21.15.191:6789/0 conn(0x7f3d940b9410 legacy=0x7f3d9406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.958+0000 7f3d99099640 1 -- 172.21.15.148:0/1543589744 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940d2ca0 con 0x7f3d940d34a0 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d99099640 1 --2- 172.21.15.148:0/1543589744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940d34a0 0x7f3d9406c0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d9406dad0 tx=0x7f3d8402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec042e2b5cd95f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d98898640 1 -- 172.21.15.148:0/1543589744 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8402e070 con 0x7f3d940d34a0 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d98898640 1 -- 172.21.15.148:0/1543589744 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84037070 con 0x7f3d940d34a0 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d98898640 1 -- 172.21.15.148:0/1543589744 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84031a00 con 0x7f3d940d34a0 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1543589744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940d34a0 msgr2=0x7f3d9406c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.959+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1543589744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940d34a0 0x7f3d9406c0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d9406dad0 tx=0x7f3d8402b880 comp rx=0 tx=0).stop 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1543589744 shutdown_connections 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1543589744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940d34a0 0x7f3d9406c0e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1543589744 >> v2:172.21.15.191:3300/0 conn(0x7f3d940b5700 0x7f3d940d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1543589744 >> 172.21.15.148:0/1543589744 conn(0x7f3d940d0490 msgr2=0x7f3d940d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1543589744 shutdown_connections 2024-03-04T12:12:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.960+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1543589744 wait complete. 2024-03-04T12:12:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 Processor -- start 2024-03-04T12:12:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 -- start start 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 0x7f3d94157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 0x7f3d941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9406ceb0 con 0x7f3d940b5700 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.961+0000 7f3d9bafe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9406cd30 con 0x7f3d940b9410 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9bafe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940b5ae0 con 0x7f3d940d34a0 2024-03-04T12:12:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39676/0 (socket says 172.21.15.148:39676) 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 -- 172.21.15.148:0/1775675815 learned_addr learned my addr 172.21.15.148:0/1775675815 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d99099640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 0x7f3d94157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9a09b640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 0x7f3d941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 -- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 msgr2=0x7f3d941587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 0x7f3d941587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 -- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 msgr2=0x7f3d94157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 0x7f3d94157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 -- 172.21.15.148:0/1775675815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940c7aa0 con 0x7f3d940b5700 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d99099640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 0x7f3d94157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9a09b640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 0x7f3d941587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9989a640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000d800 tx=0x7f3d9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000b6c0 con 0x7f3d940b5700 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d90019070 con 0x7f3d940b5700 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d940c7d80 con 0x7f3d940b5700 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000ba80 con 0x7f3d940b5700 2024-03-04T12:12:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.962+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d940c9cc0 con 0x7f3d940b5700 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.963+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d94074f60 con 0x7f3d940b5700 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.964+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3d9000bc20 con 0x7f3d940b5700 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.964+0000 7f3d8a7fc640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 0x7f3d7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.964+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d9008a960 con 0x7f3d940b5700 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.964+0000 7f3d99099640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 0x7f3d7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:14.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.965+0000 7f3d99099640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 0x7f3d7806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d84036000 tx=0x7f3d8403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:14.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:14.968+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d90059530 con 0x7f3d940b5700 2024-03-04T12:12:15.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.175+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3d940b6120 con 0x7f3d940b5700 2024-03-04T12:12:15.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.958+0000 7f3d8a7fc640 1 -- 172.21.15.148:0/1775675815 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3d90058e30 con 0x7f3d940b5700 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 msgr2=0x7f3d7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 0x7f3d7806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d84036000 tx=0x7f3d8403d040 comp rx=0 tx=0).stop 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 msgr2=0x7f3d940c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000d800 tx=0x7f3d9000dcc0 comp rx=0 tx=0).stop 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 shutdown_connections 2024-03-04T12:12:15.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3d7806b4d0 0x7f3d7806d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d940d34a0 0x7f3d941587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d940b9410 0x7f3d94157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 --2- 172.21.15.148:0/1775675815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d940b5700 0x7f3d940c9780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 >> 172.21.15.148:0/1775675815 conn(0x7f3d940d0490 msgr2=0x7f3d941601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 shutdown_connections 2024-03-04T12:12:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:15.961+0000 7f3d9bafe640 1 -- 172.21.15.148:0/1775675815 wait complete. 2024-03-04T12:12:15.975 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-04T12:12:15.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-04T12:12:16.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.083+0000 7fabcb297640 1 Processor -- start 2024-03-04T12:12:16.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.083+0000 7fabcb297640 1 -- start start 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabcb297640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fabc4161df0 0x7fabc40cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabcb297640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc4162450 con 0x7fabc40cf660 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabcb297640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40cc2d0 con 0x7fabc415da20 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabcb297640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc40d0000 con 0x7fabc4161df0 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabc8832640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fabc40cf660 0x7fabc41616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47294/0 (socket says 172.21.15.148:47294) 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabc8832640 1 -- 172.21.15.148:0/1239090280 learned_addr learned my addr 172.21.15.148:0/1239090280 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabc9834640 1 --2- 172.21.15.148:0/1239090280 >> v2:172.21.15.191:3300/0 conn(0x7fabc4161df0 0x7fabc40cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397494450 0 0) 0x7fabc4162450 con 0x7fabc40cf660 2024-03-04T12:12:16.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.084+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb0003520 con 0x7fabc40cf660 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655861791 0 0) 0x7fabb0003520 con 0x7fabc40cf660 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 >> v2:172.21.15.191:3300/0 conn(0x7fabc4161df0 msgr2=0x7fabc40cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 --2- 172.21.15.148:0/1239090280 >> v2:172.21.15.191:3300/0 conn(0x7fabc4161df0 0x7fabc40cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 >> v1:172.21.15.191:6789/0 conn(0x7fabc415da20 legacy=0x7fabc40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40cc5b0 con 0x7fabc40cf660 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fabb4001670 con 0x7fabc40cf660 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 >> v1:172.21.15.148:6789/0 conn(0x7fabc40cf660 legacy=0x7fabc41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb00039b0 0x7fabb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 0x7fabb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabb00051c0 0x7fabb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000c130 con 0x7fabb0004800 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000c2e0 con 0x7fabb00039b0 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000c4e0 con 0x7fabb00051c0 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 0x7fabb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc9033640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb00039b0 0x7fabb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc9834640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabb00051c0 0x7fabb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 -- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabb00051c0 msgr2=0x7fabb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabb00051c0 0x7fabb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 -- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb00039b0 msgr2=0x7fabb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb00039b0 0x7fabb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc8832640 1 -- 172.21.15.148:0/1239090280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0003520 con 0x7fabb0004800 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.085+0000 7fabc9834640 1 -- 172.21.15.148:0/1239090280 reap_dead start 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.086+0000 7fabc8832640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 0x7fabb0004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c240 tx=0x7fabb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c530deef82385a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.086+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4007d10 con 0x7fabb0004800 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.086+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4030070 con 0x7fabb0004800 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.086+0000 7fabbbfff640 1 -- 172.21.15.148:0/1239090280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005d90 con 0x7fabb0004800 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 -- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 msgr2=0x7fabb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 0x7fabb0004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c240 tx=0x7fabb0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:16.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 -- 172.21.15.148:0/1239090280 shutdown_connections 2024-03-04T12:12:16.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 --2- 172.21.15.148:0/1239090280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabb0004800 0x7fabb0004c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 -- 172.21.15.148:0/1239090280 >> 172.21.15.148:0/1239090280 conn(0x7fabc40b33a0 msgr2=0x7fabc40b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:16.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 -- 172.21.15.148:0/1239090280 shutdown_connections 2024-03-04T12:12:16.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.087+0000 7fabcb297640 1 -- 172.21.15.148:0/1239090280 wait complete. 2024-03-04T12:12:16.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.088+0000 7fabcb297640 1 Processor -- start 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.088+0000 7fabcb297640 1 -- start start 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb0004800 0x7fabc4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc40ad8c0 0x7fabc40add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415de00 con 0x7fabc4160a40 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40cc920 con 0x7fabb0004800 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40cfac0 con 0x7fabc40ad8c0 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39694/0 (socket says 172.21.15.148:39694) 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 -- 172.21.15.148:0/974736713 learned_addr learned my addr 172.21.15.148:0/974736713 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc9033640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb0004800 0x7fabc4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc9834640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc40ad8c0 0x7fabc40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 -- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc40ad8c0 msgr2=0x7fabc40add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc40ad8c0 0x7fabc40add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 -- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb0004800 msgr2=0x7fabc4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb0004800 0x7fabc4155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 -- 172.21.15.148:0/974736713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4160ec0 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabc8832640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001970 tx=0x7fabc4162730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4045070 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb40049f0 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc40c9f80 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.089+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4007d10 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.090+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc40ca4e0 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.090+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4074f60 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.091+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fabb4007070 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.091+0000 7fabb9ffb640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.091+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fabb40ab0c0 con 0x7fabc4160a40 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.091+0000 7fabc9033640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:16.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.092+0000 7fabc9033640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 0x7faba806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fabc0000c00 tx=0x7fabc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:16.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.096+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabc40c9f80 con 0x7fabc4160a40 2024-03-04T12:12:16.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.301+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fabc415cd20 con 0x7fabc4160a40 2024-03-04T12:12:16.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.962+0000 7fabb9ffb640 1 -- 172.21.15.148:0/974736713 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fabb4079640 con 0x7fabc4160a40 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 msgr2=0x7faba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 0x7faba806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fabc0000c00 tx=0x7fabc0009040 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 msgr2=0x7fabc40c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001970 tx=0x7fabc4162730 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 shutdown_connections 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc40ad8c0 0x7fabc40add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc4160a40 0x7fabc40c9a40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 --2- 172.21.15.148:0/974736713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabb0004800 0x7fabc4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 >> 172.21.15.148:0/974736713 conn(0x7fabc40b33a0 msgr2=0x7fabc40b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 shutdown_connections 2024-03-04T12:12:16.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:16.965+0000 7fabcb297640 1 -- 172.21.15.148:0/974736713 wait complete. 2024-03-04T12:12:16.979 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-04T12:12:16.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.083+0000 7f04e3716640 1 Processor -- start 2024-03-04T12:12:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 -- start start 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04dc06f360 0x7f04dc06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06f7c0 0x7f04dc06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04dc0b3ff0 con 0x7f04dc06acd0 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04dc06bb30 con 0x7f04dc06f360 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e3716640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc15df60 con 0x7f04dc06f7c0 2024-03-04T12:12:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e0cb1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04dc06f360 0x7f04dc06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.084+0000 7f04e14b2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f04dc06acd0 0x7f04dc0b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47304/0 (socket says 172.21.15.148:47304) 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e14b2640 1 -- 172.21.15.148:0/1862484255 learned_addr learned my addr 172.21.15.148:0/1862484255 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e1cb3640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06f7c0 0x7f04dc06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e0cb1640 1 -- 172.21.15.148:0/1862484255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06f7c0 msgr2=0x7f04dc06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e0cb1640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06f7c0 0x7f04dc06b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e0cb1640 1 -- 172.21.15.148:0/1862484255 >> v1:172.21.15.148:6789/0 conn(0x7f04dc06acd0 legacy=0x7f04dc0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e0cb1640 1 -- 172.21.15.148:0/1862484255 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc06e2c0 con 0x7f04dc06f360 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.085+0000 7f04e0cb1640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f360 0x7f04dc06b0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0097a0 tx=0x7f04cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf8d3f154639055 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04d3fff640 1 -- 172.21.15.148:0/1862484255 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc035020 con 0x7f04dc06f360 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04d3fff640 1 -- 172.21.15.148:0/1862484255 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc031830 con 0x7f04dc06f360 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04d3fff640 1 -- 172.21.15.148:0/1862484255 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc031c00 con 0x7f04dc06f360 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 -- 172.21.15.148:0/1862484255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f360 msgr2=0x7f04dc06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f360 0x7f04dc06b0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0097a0 tx=0x7f04cc02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 -- 172.21.15.148:0/1862484255 shutdown_connections 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06f7c0 0x7f04dc06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 --2- 172.21.15.148:0/1862484255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f360 0x7f04dc06b0b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 -- 172.21.15.148:0/1862484255 >> 172.21.15.148:0/1862484255 conn(0x7f04dc0cfe90 msgr2=0x7f04dc0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 -- 172.21.15.148:0/1862484255 shutdown_connections 2024-03-04T12:12:17.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.086+0000 7f04e3716640 1 -- 172.21.15.148:0/1862484255 wait complete. 2024-03-04T12:12:17.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.087+0000 7f04e3716640 1 Processor -- start 2024-03-04T12:12:17.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 -- start start 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06acd0 0x7f04dc0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04dc06f360 0x7f04dc147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06eb60 con 0x7f04dc06f7c0 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06e880 con 0x7f04dc06f360 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e3716640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06ee40 con 0x7f04dc06acd0 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e1cb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e1cb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39704/0 (socket says 172.21.15.148:39704) 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e1cb3640 1 -- 172.21.15.148:0/677821986 learned_addr learned my addr 172.21.15.148:0/677821986 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e14b2640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06acd0 0x7f04dc0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.088+0000 7f04e0cb1640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04dc06f360 0x7f04dc147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 -- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06acd0 msgr2=0x7f04dc0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06acd0 0x7f04dc0cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 -- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04dc06f360 msgr2=0x7f04dc147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04dc06f360 0x7f04dc147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 -- 172.21.15.148:0/677821986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0ce4e0 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e1cb3640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007e40 tx=0x7f04d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4014620 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4019070 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc0cee30 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4014a20 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.089+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc148810 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.090+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc074f60 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.091+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f04d4014bc0 con 0x7f04dc06f7c0 2024-03-04T12:12:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.091+0000 7f04d1ffb640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 0x7f04c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.091+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f04d408af10 con 0x7f04dc06f7c0 2024-03-04T12:12:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.091+0000 7f04e14b2640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 0x7f04c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.091+0000 7f04e14b2640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 0x7f04c006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000f60 tx=0x7f04d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.095+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04d40599c0 con 0x7f04dc06f7c0 2024-03-04T12:12:17.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.301+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04dc06b390 con 0x7f04dc06f7c0 2024-03-04T12:12:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.302+0000 7f04d1ffb640 1 -- 172.21.15.148:0/677821986 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f04d40592c0 con 0x7f04dc06f7c0 2024-03-04T12:12:17.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:17.303 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:17.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 msgr2=0x7f04c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 0x7f04c006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000f60 tx=0x7f04d8009040 comp rx=0 tx=0).stop 2024-03-04T12:12:17.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 msgr2=0x7f04dc158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007e40 tx=0x7f04d400e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 shutdown_connections 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f04c006b4d0 0x7f04c006d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04dc06f7c0 0x7f04dc158c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04dc06f360 0x7f04dc147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 --2- 172.21.15.148:0/677821986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04dc06acd0 0x7f04dc0cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 >> 172.21.15.148:0/677821986 conn(0x7f04dc0cfe90 msgr2=0x7f04dc06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 shutdown_connections 2024-03-04T12:12:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.304+0000 7f04e3716640 1 -- 172.21.15.148:0/677821986 wait complete. 2024-03-04T12:12:17.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:17.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.415+0000 7f6cbcc68640 1 Processor -- start 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.415+0000 7f6cbcc68640 1 -- start start 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cbcc68640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6cb80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cbcc68640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6cb80cfeb0 0x7f6cb81618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cbcc68640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb80cf200 con 0x7f6cb8161de0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cbcc68640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb80cc0c0 con 0x7f6cb80cfeb0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cbcc68640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8162230 con 0x7f6cb815dcd0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cb6d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6cb8161de0 0x7f6cb80d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47312/0 (socket says 172.21.15.148:47312) 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.416+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/777760015 learned_addr learned my addr 172.21.15.148:0/777760015 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6caffff640 1 --2- 172.21.15.148:0/777760015 >> v2:172.21.15.191:3300/0 conn(0x7f6cb80cfeb0 0x7f6cb81618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb659c640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6cb80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589041002 0 0) 0x7f6cb80cf200 con 0x7f6cb8161de0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca4003520 con 0x7f6cb8161de0 2024-03-04T12:12:17.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975929537 0 0) 0x7f6ca4003520 con 0x7f6cb8161de0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 msgr2=0x7f6cb80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6cb80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 >> v2:172.21.15.191:3300/0 conn(0x7f6cb80cfeb0 msgr2=0x7f6cb81618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 --2- 172.21.15.148:0/777760015 >> v2:172.21.15.191:3300/0 conn(0x7f6cb80cfeb0 0x7f6cb81618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80ce680 con 0x7f6cb8161de0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f6cb00019f0 con 0x7f6cb8161de0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 >> v1:172.21.15.148:6789/0 conn(0x7f6cb8161de0 legacy=0x7f6cb80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ca40039b0 0x7f6ca4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6ca4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca4004d40 0x7f6ca40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8162230 con 0x7f6ca4004d40 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca400b9c0 con 0x7f6ca40039b0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca400bb40 con 0x7f6cb815dcd0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca4004d40 0x7f6ca40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb659c640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ca40039b0 0x7f6ca4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6caffff640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6ca4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 msgr2=0x7f6ca4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb815dcd0 0x7f6ca4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ca40039b0 msgr2=0x7f6ca4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ca40039b0 0x7f6ca4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.417+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/777760015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4003520 con 0x7f6ca4004d40 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.418+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca4004d40 0x7f6ca40051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000a920 tx=0x7f6ca4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deccca1ab2a1b1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.418+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000a4c0 con 0x7f6ca4004d40 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.418+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0034650 con 0x7f6ca4004d40 2024-03-04T12:12:17.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.418+0000 7f6cb5d9b640 1 -- 172.21.15.148:0/777760015 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0034a40 con 0x7f6ca4004d40 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 -- 172.21.15.148:0/777760015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca4004d40 msgr2=0x7f6ca40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/777760015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca4004d40 0x7f6ca40051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000a920 tx=0x7f6ca4005830 comp rx=0 tx=0).stop 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/777760015 reap_dead start 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 -- 172.21.15.148:0/777760015 shutdown_connections 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 -- 172.21.15.148:0/777760015 >> 172.21.15.148:0/777760015 conn(0x7f6cb80b3480 msgr2=0x7f6cb80b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 -- 172.21.15.148:0/777760015 shutdown_connections 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.419+0000 7f6cbcc68640 1 -- 172.21.15.148:0/777760015 wait complete. 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.420+0000 7f6cbcc68640 1 Processor -- start 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.420+0000 7f6cbcc68640 1 -- start start 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 0x7f6cb80c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80c60d0 0x7f6cb80c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb80cef20 con 0x7f6cb80c94a0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb80cec40 con 0x7f6cb8151ed0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cbcc68640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb806bff0 con 0x7f6cb80c60d0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39718/0 (socket says 172.21.15.148:39718) 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/782959375 learned_addr learned my addr 172.21.15.148:0/782959375 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6caffff640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80c60d0 0x7f6cb80c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb659c640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 0x7f6cb80c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80c60d0 msgr2=0x7f6cb80c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80c60d0 0x7f6cb80c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 msgr2=0x7f6cb80c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 0x7f6cb80c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 -- 172.21.15.148:0/782959375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80c6e50 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb659c640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 0x7f6cb80c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.421+0000 7f6cb6d9d640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0001cf0 tx=0x7f6cb80ce850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.422+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000a4c0 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.422+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0045070 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.422+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0002000 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.422+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb80c65b0 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.422+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb80c6b70 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.423+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb8074f60 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.423+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6cb0034c00 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.423+0000 7f6caeffd640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 0x7f6c9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.424+0000 7f6cb659c640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 0x7f6c9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.424+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6cb00ac970 con 0x7f6cb80c94a0 2024-03-04T12:12:17.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.424+0000 7f6cb659c640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 0x7f6c9806d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009a40 tx=0x7f6ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.428+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cb0032380 con 0x7f6cb80c94a0 2024-03-04T12:12:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.601+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cb815c240 con 0x7f6cb80c94a0 2024-03-04T12:12:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.602+0000 7f6caeffd640 1 -- 172.21.15.148:0/782959375 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f6cb007b540 con 0x7f6cb80c94a0 2024-03-04T12:12:17.604 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:17.604 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 msgr2=0x7f6c9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 0x7f6c9806d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009a40 tx=0x7f6ca8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 msgr2=0x7f6cb80c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0001cf0 tx=0x7f6cb80ce850 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 shutdown_connections 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6c9806b500 0x7f6c9806d9c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80c94a0 0x7f6cb80c9950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80c60d0 0x7f6cb80c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 --2- 172.21.15.148:0/782959375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb8151ed0 0x7f6cb80c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 >> 172.21.15.148:0/782959375 conn(0x7f6cb80b3480 msgr2=0x7f6cb80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 shutdown_connections 2024-03-04T12:12:17.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.604+0000 7f6cbcc68640 1 -- 172.21.15.148:0/782959375 wait complete. 2024-03-04T12:12:17.617 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-04T12:12:17.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:12:17.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.717+0000 7f3b74ba2640 1 Processor -- start 2024-03-04T12:12:17.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.717+0000 7f3b74ba2640 1 -- start start 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.717+0000 7f3b74ba2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf030 0x7f3b7006f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.717+0000 7f3b74ba2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b700669a0 0x7f3b700acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b74ba2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7015df90 con 0x7f3b700cf410 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b74ba2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b70067110 con 0x7f3b700669a0 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b74ba2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b70066e30 con 0x7f3b700cf030 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b700669a0 0x7f3b700acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3b700cf410 0x7f3b7006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47320/0 (socket says 172.21.15.148:47320) 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6ffff640 1 -- 172.21.15.148:0/1800383473 learned_addr learned my addr 172.21.15.148:0/1800383473 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:17.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6f7fe640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf030 0x7f3b7006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6e7fc640 1 -- 172.21.15.148:0/1800383473 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105961996 0 0) 0x7f3b7015df90 con 0x7f3b700cf410 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6e7fc640 1 -- 172.21.15.148:0/1800383473 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b58003520 con 0x7f3b700cf410 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6effd640 1 -- 172.21.15.148:0/1800383473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf030 msgr2=0x7f3b7006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6effd640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf030 0x7f3b7006f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6effd640 1 -- 172.21.15.148:0/1800383473 >> v1:172.21.15.148:6789/0 conn(0x7f3b700cf410 legacy=0x7f3b7006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.718+0000 7f3b6effd640 1 -- 172.21.15.148:0/1800383473 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7006ae00 con 0x7f3b700669a0 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b6effd640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700669a0 0x7f3b700acd40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b640097a0 tx=0x7f3b6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1ad9898717a11f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b6e7fc640 1 -- 172.21.15.148:0/1800383473 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b640315a0 con 0x7f3b700669a0 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b6e7fc640 1 -- 172.21.15.148:0/1800383473 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64037070 con 0x7f3b700669a0 2024-03-04T12:12:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b6e7fc640 1 -- 172.21.15.148:0/1800383473 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64031960 con 0x7f3b700669a0 2024-03-04T12:12:17.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1800383473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700669a0 msgr2=0x7f3b700acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700669a0 0x7f3b700acd40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b640097a0 tx=0x7f3b6402b840 comp rx=0 tx=0).stop 2024-03-04T12:12:17.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1800383473 shutdown_connections 2024-03-04T12:12:17.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700669a0 0x7f3b700acd40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1800383473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf030 0x7f3b7006f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.719+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1800383473 >> 172.21.15.148:0/1800383473 conn(0x7f3b700cfca0 msgr2=0x7f3b700d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.720+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1800383473 shutdown_connections 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.720+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1800383473 wait complete. 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.721+0000 7f3b74ba2640 1 Processor -- start 2024-03-04T12:12:17.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.721+0000 7f3b74ba2640 1 -- start start 2024-03-04T12:12:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.721+0000 7f3b74ba2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 0x7f3b701489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.721+0000 7f3b74ba2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf410 0x7f3b700b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700af5a0 con 0x7f3b700b06f0 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7006b3c0 con 0x7f3b700cf030 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700af720 con 0x7f3b700cf410 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39750/0 (socket says 172.21.15.148:39750) 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 -- 172.21.15.148:0/1839745102 learned_addr learned my addr 172.21.15.148:0/1839745102 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6effd640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf410 0x7f3b700b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6f7fe640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 0x7f3b701489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 -- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf410 msgr2=0x7f3b700b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf410 0x7f3b700b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 -- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 msgr2=0x7f3b701489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 0x7f3b701489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 -- 172.21.15.148:0/1839745102 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7015a5f0 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6f7fe640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 0x7f3b701489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b6ffff640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800c400 tx=0x7f3b6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68017070 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68005ce0 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b680072d0 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b700b2ac0 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.722+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b700b3080 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.723+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7004f100 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.724+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3b6801a070 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.724+0000 7f3b57fff640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 0x7f3b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.724+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b68089800 con 0x7f3b700b06f0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.724+0000 7f3b6f7fe640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 0x7f3b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.725+0000 7f3b6f7fe640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 0x7f3b5c06d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009b20 tx=0x7f3b600039d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:17.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.728+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b68057cf0 con 0x7f3b700b06f0 2024-03-04T12:12:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.898+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b700cc190 con 0x7f3b5c06b4d0 2024-03-04T12:12:17.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.906+0000 7f3b57fff640 1 -- 172.21.15.148:0/1839745102 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f3b700cc190 con 0x7f3b5c06b4d0 2024-03-04T12:12:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 msgr2=0x7f3b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 0x7f3b5c06d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009b20 tx=0x7f3b600039d0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 msgr2=0x7f3b700b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800c400 tx=0x7f3b6800c8c0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 shutdown_connections 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3b5c06b4d0 0x7f3b5c06d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b700b06f0 0x7f3b700b0ba0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b700cf410 0x7f3b700b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 --2- 172.21.15.148:0/1839745102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b700cf030 0x7f3b701489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:17.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 >> 172.21.15.148:0/1839745102 conn(0x7f3b700cfca0 msgr2=0x7f3b701546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:17.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 shutdown_connections 2024-03-04T12:12:17.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:17.910+0000 7f3b74ba2640 1 -- 172.21.15.148:0/1839745102 wait complete. 2024-03-04T12:12:17.922 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-03-04T12:12:16.718598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771745+0000","last_change":"2024-03-04T12:11:53.892196+0000","last_active":"2024-03-04T12:12:00.771745+0000","last_peered":"2024-03-04T12:12:00.771745+0000","last_clean":"2024-03-04T12:12:00.771745+0000","last_became_active":"2024-03-04T12:11:53.891878+0000","last_became_peered":"2024-03-04T12:11:53.891878+0000","last_unstale":"2024-03-04T12:12:00.771745+0000","last_undegraded":"2024-03-04T12:12:00.771745+0000","last_fullsized":"2024-03-04T12:12:00.771745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:54.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921065+0000","last_change":"2024-03-04T12:11:54.905432+0000","last_active":"2024-03-04T12:11:59.921065+0000","last_peered":"2024-03-04T12:11:59.921065+0000","last_clean":"2024-03-04T12:11:59.921065+0000","last_became_active":"2024-03-04T12:11:54.905131+0000","last_became_peered":"2024-03-04T12:11:54.905131+0000","last_unstale":"2024-03-04T12:11:59.921065+0000","last_undegraded":"2024-03-04T12:11:59.921065+0000","last_fullsized":"2024-03-04T12:11:59.921065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920122+0000","last_change":"2024-03-04T12:11:58.938162+0000","last_active":"2024-03-04T12:11:59.920122+0000","last_peered":"2024-03-04T12:11:59.920122+0000","last_clean":"2024-03-04T12:11:59.920122+0000","last_became_active":"2024-03-04T12:11:58.937622+0000","last_became_peered":"2024-03-04T12:11:58.937622+0000","last_unstale":"2024-03-04T12:11:59.920122+0000","last_undegraded":"2024-03-04T12:11:59.920122+0000","last_fullsized":"2024-03-04T12:11:59.920122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:33:55.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921215+0000","last_change":"2024-03-04T12:11:53.881535+0000","last_active":"2024-03-04T12:11:59.921215+0000","last_peered":"2024-03-04T12:11:59.921215+0000","last_clean":"2024-03-04T12:11:59.921215+0000","last_became_active":"2024-03-04T12:11:53.880591+0000","last_became_peered":"2024-03-04T12:11:53.880591+0000","last_unstale":"2024-03-04T12:11:59.921215+0000","last_undegraded":"2024-03-04T12:11:59.921215+0000","last_fullsized":"2024-03-04T12:11:59.921215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771443+0000","last_change":"2024-03-04T12:11:54.906418+0000","last_active":"2024-03-04T12:12:00.771443+0000","last_peered":"2024-03-04T12:12:00.771443+0000","last_clean":"2024-03-04T12:12:00.771443+0000","last_became_active":"2024-03-04T12:11:54.906146+0000","last_became_peered":"2024-03-04T12:11:54.906146+0000","last_unstale":"2024-03-04T12:12:00.771443+0000","last_undegraded":"2024-03-04T12:12:00.771443+0000","last_fullsized":"2024-03-04T12:12:00.771443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771197+0000","last_change":"2024-03-04T12:11:58.945630+0000","last_active":"2024-03-04T12:12:00.771197+0000","last_peered":"2024-03-04T12:12:00.771197+0000","last_clean":"2024-03-04T12:12:00.771197+0000","last_became_active":"2024-03-04T12:11:58.945230+0000","last_became_peered":"2024-03-04T12:11:58.945230+0000","last_unstale":"2024-03-04T12:12:00.771197+0000","last_undegraded":"2024-03-04T12:12:00.771197+0000","last_fullsized":"2024-03-04T12:12:00.771197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:50.652152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.895431+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.895122+0000","last_became_peered":"2024-03-04T12:11:53.895122+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:54.330076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771997+0000","last_change":"2024-03-04T12:11:54.902410+0000","last_active":"2024-03-04T12:12:00.771997+0000","last_peered":"2024-03-04T12:12:00.771997+0000","last_clean":"2024-03-04T12:12:00.771997+0000","last_became_active":"2024-03-04T12:11:54.902131+0000","last_became_peered":"2024-03-04T12:11:54.902131+0000","last_unstale":"2024-03-04T12:12:00.771997+0000","last_undegraded":"2024-03-04T12:12:00.771997+0000","last_fullsized":"2024-03-04T12:12:00.771997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772908+0000","last_change":"2024-03-04T12:11:58.943323+0000","last_active":"2024-03-04T12:12:00.772908+0000","last_peered":"2024-03-04T12:12:00.772908+0000","last_clean":"2024-03-04T12:12:00.772908+0000","last_became_active":"2024-03-04T12:11:58.942878+0000","last_became_peered":"2024-03-04T12:11:58.942878+0000","last_unstale":"2024-03-04T12:12:00.772908+0000","last_undegraded":"2024-03-04T12:12:00.772908+0000","last_fullsized":"2024-03-04T12:12:00.772908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771622+0000","last_change":"2024-03-04T12:11:53.893583+0000","last_active":"2024-03-04T12:12:00.771622+0000","last_peered":"2024-03-04T12:12:00.771622+0000","last_clean":"2024-03-04T12:12:00.771622+0000","last_became_active":"2024-03-04T12:11:53.893295+0000","last_became_peered":"2024-03-04T12:11:53.893295+0000","last_unstale":"2024-03-04T12:12:00.771622+0000","last_undegraded":"2024-03-04T12:12:00.771622+0000","last_fullsized":"2024-03-04T12:12:00.771622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772748+0000","last_change":"2024-03-04T12:11:54.901339+0000","last_active":"2024-03-04T12:12:00.772748+0000","last_peered":"2024-03-04T12:12:00.772748+0000","last_clean":"2024-03-04T12:12:00.772748+0000","last_became_active":"2024-03-04T12:11:54.900914+0000","last_became_peered":"2024-03-04T12:11:54.900914+0000","last_unstale":"2024-03-04T12:12:00.772748+0000","last_undegraded":"2024-03-04T12:12:00.772748+0000","last_fullsized":"2024-03-04T12:12:00.772748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773092+0000","last_change":"2024-03-04T12:11:58.937633+0000","last_active":"2024-03-04T12:12:00.773092+0000","last_peered":"2024-03-04T12:12:00.773092+0000","last_clean":"2024-03-04T12:12:00.773092+0000","last_became_active":"2024-03-04T12:11:58.936727+0000","last_became_peered":"2024-03-04T12:11:58.936727+0000","last_unstale":"2024-03-04T12:12:00.773092+0000","last_undegraded":"2024-03-04T12:12:00.773092+0000","last_fullsized":"2024-03-04T12:12:00.773092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:03:00.255209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921104+0000","last_change":"2024-03-04T12:11:54.905529+0000","last_active":"2024-03-04T12:11:59.921104+0000","last_peered":"2024-03-04T12:11:59.921104+0000","last_clean":"2024-03-04T12:11:59.921104+0000","last_became_active":"2024-03-04T12:11:54.904590+0000","last_became_peered":"2024-03-04T12:11:54.904590+0000","last_unstale":"2024-03-04T12:11:59.921104+0000","last_undegraded":"2024-03-04T12:11:59.921104+0000","last_fullsized":"2024-03-04T12:11:59.921104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772191+0000","last_change":"2024-03-04T12:11:53.892967+0000","last_active":"2024-03-04T12:12:00.772191+0000","last_peered":"2024-03-04T12:12:00.772191+0000","last_clean":"2024-03-04T12:12:00.772191+0000","last_became_active":"2024-03-04T12:11:53.892698+0000","last_became_peered":"2024-03-04T12:11:53.892698+0000","last_unstale":"2024-03-04T12:12:00.772191+0000","last_undegraded":"2024-03-04T12:12:00.772191+0000","last_fullsized":"2024-03-04T12:12:00.772191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772796+0000","last_change":"2024-03-04T12:11:58.947133+0000","last_active":"2024-03-04T12:12:00.772796+0000","last_peered":"2024-03-04T12:12:00.772796+0000","last_clean":"2024-03-04T12:12:00.772796+0000","last_became_active":"2024-03-04T12:11:58.946617+0000","last_became_peered":"2024-03-04T12:11:58.946617+0000","last_unstale":"2024-03-04T12:12:00.772796+0000","last_undegraded":"2024-03-04T12:12:00.772796+0000","last_fullsized":"2024-03-04T12:12:00.772796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920487+0000","last_change":"2024-03-04T12:11:54.896623+0000","last_active":"2024-03-04T12:11:59.920487+0000","last_peered":"2024-03-04T12:11:59.920487+0000","last_clean":"2024-03-04T12:11:59.920487+0000","last_became_active":"2024-03-04T12:11:54.896359+0000","last_became_peered":"2024-03-04T12:11:54.896359+0000","last_unstale":"2024-03-04T12:11:59.920487+0000","last_undegraded":"2024-03-04T12:11:59.920487+0000","last_fullsized":"2024-03-04T12:11:59.920487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771403+0000","last_change":"2024-03-04T12:11:53.886825+0000","last_active":"2024-03-04T12:12:00.771403+0000","last_peered":"2024-03-04T12:12:00.771403+0000","last_clean":"2024-03-04T12:12:00.771403+0000","last_became_active":"2024-03-04T12:11:53.886525+0000","last_became_peered":"2024-03-04T12:11:53.886525+0000","last_unstale":"2024-03-04T12:12:00.771403+0000","last_undegraded":"2024-03-04T12:12:00.771403+0000","last_fullsized":"2024-03-04T12:12:00.771403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772607+0000","last_change":"2024-03-04T12:11:58.946327+0000","last_active":"2024-03-04T12:12:00.772607+0000","last_peered":"2024-03-04T12:12:00.772607+0000","last_clean":"2024-03-04T12:12:00.772607+0000","last_became_active":"2024-03-04T12:11:58.945962+0000","last_became_peered":"2024-03-04T12:11:58.945962+0000","last_unstale":"2024-03-04T12:12:00.772607+0000","last_undegraded":"2024-03-04T12:12:00.772607+0000","last_fullsized":"2024-03-04T12:12:00.772607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920261+0000","last_change":"2024-03-04T12:11:54.903139+0000","last_active":"2024-03-04T12:11:59.920261+0000","last_peered":"2024-03-04T12:11:59.920261+0000","last_clean":"2024-03-04T12:11:59.920261+0000","last_became_active":"2024-03-04T12:11:54.902809+0000","last_became_peered":"2024-03-04T12:11:54.902809+0000","last_unstale":"2024-03-04T12:11:59.920261+0000","last_undegraded":"2024-03-04T12:11:59.920261+0000","last_fullsized":"2024-03-04T12:11:59.920261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770887+0000","last_change":"2024-03-04T12:11:53.889802+0000","last_active":"2024-03-04T12:12:00.770887+0000","last_peered":"2024-03-04T12:12:00.770887+0000","last_clean":"2024-03-04T12:12:00.770887+0000","last_became_active":"2024-03-04T12:11:53.889550+0000","last_became_peered":"2024-03-04T12:11:53.889550+0000","last_unstale":"2024-03-04T12:12:00.770887+0000","last_undegraded":"2024-03-04T12:12:00.770887+0000","last_fullsized":"2024-03-04T12:12:00.770887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771267+0000","last_change":"2024-03-04T12:11:58.941310+0000","last_active":"2024-03-04T12:12:00.771267+0000","last_peered":"2024-03-04T12:12:00.771267+0000","last_clean":"2024-03-04T12:12:00.771267+0000","last_became_active":"2024-03-04T12:11:58.940932+0000","last_became_peered":"2024-03-04T12:11:58.940932+0000","last_unstale":"2024-03-04T12:12:00.771267+0000","last_undegraded":"2024-03-04T12:12:00.771267+0000","last_fullsized":"2024-03-04T12:12:00.771267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772213+0000","last_change":"2024-03-04T12:11:54.900654+0000","last_active":"2024-03-04T12:12:00.772213+0000","last_peered":"2024-03-04T12:12:00.772213+0000","last_clean":"2024-03-04T12:12:00.772213+0000","last_became_active":"2024-03-04T12:11:54.900311+0000","last_became_peered":"2024-03-04T12:11:54.900311+0000","last_unstale":"2024-03-04T12:12:00.772213+0000","last_undegraded":"2024-03-04T12:12:00.772213+0000","last_fullsized":"2024-03-04T12:12:00.772213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:23.687708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771071+0000","last_change":"2024-03-04T12:11:53.892220+0000","last_active":"2024-03-04T12:12:00.771071+0000","last_peered":"2024-03-04T12:12:00.771071+0000","last_clean":"2024-03-04T12:12:00.771071+0000","last_became_active":"2024-03-04T12:11:53.891859+0000","last_became_peered":"2024-03-04T12:11:53.891859+0000","last_unstale":"2024-03-04T12:12:00.771071+0000","last_undegraded":"2024-03-04T12:12:00.771071+0000","last_fullsized":"2024-03-04T12:12:00.771071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771968+0000","last_change":"2024-03-04T12:11:58.947661+0000","last_active":"2024-03-04T12:12:00.771968+0000","last_peered":"2024-03-04T12:12:00.771968+0000","last_clean":"2024-03-04T12:12:00.771968+0000","last_became_active":"2024-03-04T12:11:58.946889+0000","last_became_peered":"2024-03-04T12:11:58.946889+0000","last_unstale":"2024-03-04T12:12:00.771968+0000","last_undegraded":"2024-03-04T12:12:00.771968+0000","last_fullsized":"2024-03-04T12:12:00.771968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:48.995386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771351+0000","last_change":"2024-03-04T12:11:54.898183+0000","last_active":"2024-03-04T12:12:00.771351+0000","last_peered":"2024-03-04T12:12:00.771351+0000","last_clean":"2024-03-04T12:12:00.771351+0000","last_became_active":"2024-03-04T12:11:54.897873+0000","last_became_peered":"2024-03-04T12:11:54.897873+0000","last_unstale":"2024-03-04T12:12:00.771351+0000","last_undegraded":"2024-03-04T12:12:00.771351+0000","last_fullsized":"2024-03-04T12:12:00.771351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921618+0000","last_change":"2024-03-04T12:11:53.893093+0000","last_active":"2024-03-04T12:11:59.921618+0000","last_peered":"2024-03-04T12:11:59.921618+0000","last_clean":"2024-03-04T12:11:59.921618+0000","last_became_active":"2024-03-04T12:11:53.892821+0000","last_became_peered":"2024-03-04T12:11:53.892821+0000","last_unstale":"2024-03-04T12:11:59.921618+0000","last_undegraded":"2024-03-04T12:11:59.921618+0000","last_fullsized":"2024-03-04T12:11:59.921618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771969+0000","last_change":"2024-03-04T12:11:58.936275+0000","last_active":"2024-03-04T12:12:00.771969+0000","last_peered":"2024-03-04T12:12:00.771969+0000","last_clean":"2024-03-04T12:12:00.771969+0000","last_became_active":"2024-03-04T12:11:58.935879+0000","last_became_peered":"2024-03-04T12:11:58.935879+0000","last_unstale":"2024-03-04T12:12:00.771969+0000","last_undegraded":"2024-03-04T12:12:00.771969+0000","last_fullsized":"2024-03-04T12:12:00.771969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:57.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921287+0000","last_change":"2024-03-04T12:11:54.905996+0000","last_active":"2024-03-04T12:11:59.921287+0000","last_peered":"2024-03-04T12:11:59.921287+0000","last_clean":"2024-03-04T12:11:59.921287+0000","last_became_active":"2024-03-04T12:11:54.905164+0000","last_became_peered":"2024-03-04T12:11:54.905164+0000","last_unstale":"2024-03-04T12:11:59.921287+0000","last_undegraded":"2024-03-04T12:11:59.921287+0000","last_fullsized":"2024-03-04T12:11:59.921287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771364+0000","last_change":"2024-03-04T12:11:53.889023+0000","last_active":"2024-03-04T12:12:00.771364+0000","last_peered":"2024-03-04T12:12:00.771364+0000","last_clean":"2024-03-04T12:12:00.771364+0000","last_became_active":"2024-03-04T12:11:53.888732+0000","last_became_peered":"2024-03-04T12:11:53.888732+0000","last_unstale":"2024-03-04T12:12:00.771364+0000","last_undegraded":"2024-03-04T12:12:00.771364+0000","last_fullsized":"2024-03-04T12:12:00.771364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920041+0000","last_change":"2024-03-04T12:11:58.933844+0000","last_active":"2024-03-04T12:11:59.920041+0000","last_peered":"2024-03-04T12:11:59.920041+0000","last_clean":"2024-03-04T12:11:59.920041+0000","last_became_active":"2024-03-04T12:11:58.933456+0000","last_became_peered":"2024-03-04T12:11:58.933456+0000","last_unstale":"2024-03-04T12:11:59.920041+0000","last_undegraded":"2024-03-04T12:11:59.920041+0000","last_fullsized":"2024-03-04T12:11:59.920041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771789+0000","last_change":"2024-03-04T12:11:54.900251+0000","last_active":"2024-03-04T12:12:00.771789+0000","last_peered":"2024-03-04T12:12:00.771789+0000","last_clean":"2024-03-04T12:12:00.771789+0000","last_became_active":"2024-03-04T12:11:54.899988+0000","last_became_peered":"2024-03-04T12:11:54.899988+0000","last_unstale":"2024-03-04T12:12:00.771789+0000","last_undegraded":"2024-03-04T12:12:00.771789+0000","last_fullsized":"2024-03-04T12:12:00.771789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771966+0000","last_change":"2024-03-04T12:11:53.893557+0000","last_active":"2024-03-04T12:12:00.771966+0000","last_peered":"2024-03-04T12:12:00.771966+0000","last_clean":"2024-03-04T12:12:00.771966+0000","last_became_active":"2024-03-04T12:11:53.893170+0000","last_became_peered":"2024-03-04T12:11:53.893170+0000","last_unstale":"2024-03-04T12:12:00.771966+0000","last_undegraded":"2024-03-04T12:12:00.771966+0000","last_fullsized":"2024-03-04T12:12:00.771966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:04.682290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770797+0000","last_change":"2024-03-04T12:11:58.944095+0000","last_active":"2024-03-04T12:12:00.770797+0000","last_peered":"2024-03-04T12:12:00.770797+0000","last_clean":"2024-03-04T12:12:00.770797+0000","last_became_active":"2024-03-04T12:11:58.943691+0000","last_became_peered":"2024-03-04T12:11:58.943691+0000","last_unstale":"2024-03-04T12:12:00.770797+0000","last_undegraded":"2024-03-04T12:12:00.770797+0000","last_fullsized":"2024-03-04T12:12:00.770797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772840+0000","last_change":"2024-03-04T12:11:54.909256+0000","last_active":"2024-03-04T12:12:00.772840+0000","last_peered":"2024-03-04T12:12:00.772840+0000","last_clean":"2024-03-04T12:12:00.772840+0000","last_became_active":"2024-03-04T12:11:54.908971+0000","last_became_peered":"2024-03-04T12:11:54.908971+0000","last_unstale":"2024-03-04T12:12:00.772840+0000","last_undegraded":"2024-03-04T12:12:00.772840+0000","last_fullsized":"2024-03-04T12:12:00.772840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771832+0000","last_change":"2024-03-04T12:11:53.881754+0000","last_active":"2024-03-04T12:12:00.771832+0000","last_peered":"2024-03-04T12:12:00.771832+0000","last_clean":"2024-03-04T12:12:00.771832+0000","last_became_active":"2024-03-04T12:11:53.881149+0000","last_became_peered":"2024-03-04T12:11:53.881149+0000","last_unstale":"2024-03-04T12:12:00.771832+0000","last_undegraded":"2024-03-04T12:12:00.771832+0000","last_fullsized":"2024-03-04T12:12:00.771832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771018+0000","last_change":"2024-03-04T12:11:58.939152+0000","last_active":"2024-03-04T12:12:00.771018+0000","last_peered":"2024-03-04T12:12:00.771018+0000","last_clean":"2024-03-04T12:12:00.771018+0000","last_became_active":"2024-03-04T12:11:58.938820+0000","last_became_peered":"2024-03-04T12:11:58.938820+0000","last_unstale":"2024-03-04T12:12:00.771018+0000","last_undegraded":"2024-03-04T12:12:00.771018+0000","last_fullsized":"2024-03-04T12:12:00.771018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773199+0000","last_change":"2024-03-04T12:11:54.893605+0000","last_active":"2024-03-04T12:12:00.773199+0000","last_peered":"2024-03-04T12:12:00.773199+0000","last_clean":"2024-03-04T12:12:00.773199+0000","last_became_active":"2024-03-04T12:11:54.893164+0000","last_became_peered":"2024-03-04T12:11:54.893164+0000","last_unstale":"2024-03-04T12:12:00.773199+0000","last_undegraded":"2024-03-04T12:12:00.773199+0000","last_fullsized":"2024-03-04T12:12:00.773199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:32.197141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.884670+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.884396+0000","last_became_peered":"2024-03-04T12:11:53.884396+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771791+0000","last_change":"2024-03-04T12:11:58.939922+0000","last_active":"2024-03-04T12:12:00.771791+0000","last_peered":"2024-03-04T12:12:00.771791+0000","last_clean":"2024-03-04T12:12:00.771791+0000","last_became_active":"2024-03-04T12:11:58.939562+0000","last_became_peered":"2024-03-04T12:11:58.939562+0000","last_unstale":"2024-03-04T12:12:00.771791+0000","last_undegraded":"2024-03-04T12:12:00.771791+0000","last_fullsized":"2024-03-04T12:12:00.771791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770827+0000","last_change":"2024-03-04T12:11:54.898108+0000","last_active":"2024-03-04T12:12:00.770827+0000","last_peered":"2024-03-04T12:12:00.770827+0000","last_clean":"2024-03-04T12:12:00.770827+0000","last_became_active":"2024-03-04T12:11:54.897832+0000","last_became_peered":"2024-03-04T12:11:54.897832+0000","last_unstale":"2024-03-04T12:12:00.770827+0000","last_undegraded":"2024-03-04T12:12:00.770827+0000","last_fullsized":"2024-03-04T12:12:00.770827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921186+0000","last_change":"2024-03-04T12:11:53.884846+0000","last_active":"2024-03-04T12:11:59.921186+0000","last_peered":"2024-03-04T12:11:59.921186+0000","last_clean":"2024-03-04T12:11:59.921186+0000","last_became_active":"2024-03-04T12:11:53.884567+0000","last_became_peered":"2024-03-04T12:11:53.884567+0000","last_unstale":"2024-03-04T12:11:59.921186+0000","last_undegraded":"2024-03-04T12:11:59.921186+0000","last_fullsized":"2024-03-04T12:11:59.921186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771888+0000","last_change":"2024-03-04T12:11:58.936068+0000","last_active":"2024-03-04T12:12:00.771888+0000","last_peered":"2024-03-04T12:12:00.771888+0000","last_clean":"2024-03-04T12:12:00.771888+0000","last_became_active":"2024-03-04T12:11:58.935721+0000","last_became_peered":"2024-03-04T12:11:58.935721+0000","last_unstale":"2024-03-04T12:12:00.771888+0000","last_undegraded":"2024-03-04T12:12:00.771888+0000","last_fullsized":"2024-03-04T12:12:00.771888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770381+0000","last_change":"2024-03-04T12:11:54.893726+0000","last_active":"2024-03-04T12:12:00.770381+0000","last_peered":"2024-03-04T12:12:00.770381+0000","last_clean":"2024-03-04T12:12:00.770381+0000","last_became_active":"2024-03-04T12:11:54.893038+0000","last_became_peered":"2024-03-04T12:11:54.893038+0000","last_unstale":"2024-03-04T12:12:00.770381+0000","last_undegraded":"2024-03-04T12:12:00.770381+0000","last_fullsized":"2024-03-04T12:12:00.770381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770859+0000","last_change":"2024-03-04T12:11:53.893383+0000","last_active":"2024-03-04T12:12:00.770859+0000","last_peered":"2024-03-04T12:12:00.770859+0000","last_clean":"2024-03-04T12:12:00.770859+0000","last_became_active":"2024-03-04T12:11:53.892630+0000","last_became_peered":"2024-03-04T12:11:53.892630+0000","last_unstale":"2024-03-04T12:12:00.770859+0000","last_undegraded":"2024-03-04T12:12:00.770859+0000","last_fullsized":"2024-03-04T12:12:00.770859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771246+0000","last_change":"2024-03-04T12:11:58.946451+0000","last_active":"2024-03-04T12:12:00.771246+0000","last_peered":"2024-03-04T12:12:00.771246+0000","last_clean":"2024-03-04T12:12:00.771246+0000","last_became_active":"2024-03-04T12:11:58.946080+0000","last_became_peered":"2024-03-04T12:11:58.946080+0000","last_unstale":"2024-03-04T12:12:00.771246+0000","last_undegraded":"2024-03-04T12:12:00.771246+0000","last_fullsized":"2024-03-04T12:12:00.771246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920426+0000","last_change":"2024-03-04T12:11:54.902080+0000","last_active":"2024-03-04T12:11:59.920426+0000","last_peered":"2024-03-04T12:11:59.920426+0000","last_clean":"2024-03-04T12:11:59.920426+0000","last_became_active":"2024-03-04T12:11:54.901382+0000","last_became_peered":"2024-03-04T12:11:54.901382+0000","last_unstale":"2024-03-04T12:11:59.920426+0000","last_undegraded":"2024-03-04T12:11:59.920426+0000","last_fullsized":"2024-03-04T12:11:59.920426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772165+0000","last_change":"2024-03-04T12:11:53.883991+0000","last_active":"2024-03-04T12:12:00.772165+0000","last_peered":"2024-03-04T12:12:00.772165+0000","last_clean":"2024-03-04T12:12:00.772165+0000","last_became_active":"2024-03-04T12:11:53.881700+0000","last_became_peered":"2024-03-04T12:11:53.881700+0000","last_unstale":"2024-03-04T12:12:00.772165+0000","last_undegraded":"2024-03-04T12:12:00.772165+0000","last_fullsized":"2024-03-04T12:12:00.772165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770976+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:12:00.770976+0000","last_peered":"2024-03-04T12:12:00.770976+0000","last_clean":"2024-03-04T12:12:00.770976+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:12:00.770976+0000","last_undegraded":"2024-03-04T12:12:00.770976+0000","last_fullsized":"2024-03-04T12:12:00.770976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921000+0000","last_change":"2024-03-04T12:11:58.939590+0000","last_active":"2024-03-04T12:11:59.921000+0000","last_peered":"2024-03-04T12:11:59.921000+0000","last_clean":"2024-03-04T12:11:59.921000+0000","last_became_active":"2024-03-04T12:11:58.939177+0000","last_became_peered":"2024-03-04T12:11:58.939177+0000","last_unstale":"2024-03-04T12:11:59.921000+0000","last_undegraded":"2024-03-04T12:11:59.921000+0000","last_fullsized":"2024-03-04T12:11:59.921000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:54.906271+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:54.905428+0000","last_became_peered":"2024-03-04T12:11:54.905428+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921284+0000","last_change":"2024-03-04T12:11:53.885617+0000","last_active":"2024-03-04T12:11:59.921284+0000","last_peered":"2024-03-04T12:11:59.921284+0000","last_clean":"2024-03-04T12:11:59.921284+0000","last_became_active":"2024-03-04T12:11:53.885329+0000","last_became_peered":"2024-03-04T12:11:53.885329+0000","last_unstale":"2024-03-04T12:11:59.921284+0000","last_undegraded":"2024-03-04T12:11:59.921284+0000","last_fullsized":"2024-03-04T12:11:59.921284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921212+0000","last_change":"2024-03-04T12:11:58.946535+0000","last_active":"2024-03-04T12:11:59.921212+0000","last_peered":"2024-03-04T12:11:59.921212+0000","last_clean":"2024-03-04T12:11:59.921212+0000","last_became_active":"2024-03-04T12:11:58.946055+0000","last_became_peered":"2024-03-04T12:11:58.946055+0000","last_unstale":"2024-03-04T12:11:59.921212+0000","last_undegraded":"2024-03-04T12:11:59.921212+0000","last_fullsized":"2024-03-04T12:11:59.921212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772109+0000","last_change":"2024-03-04T12:11:54.910474+0000","last_active":"2024-03-04T12:12:00.772109+0000","last_peered":"2024-03-04T12:12:00.772109+0000","last_clean":"2024-03-04T12:12:00.772109+0000","last_became_active":"2024-03-04T12:11:54.909806+0000","last_became_peered":"2024-03-04T12:11:54.909806+0000","last_unstale":"2024-03-04T12:12:00.772109+0000","last_undegraded":"2024-03-04T12:12:00.772109+0000","last_fullsized":"2024-03-04T12:12:00.772109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771668+0000","last_change":"2024-03-04T12:11:53.882414+0000","last_active":"2024-03-04T12:12:00.771668+0000","last_peered":"2024-03-04T12:12:00.771668+0000","last_clean":"2024-03-04T12:12:00.771668+0000","last_became_active":"2024-03-04T12:11:53.882147+0000","last_became_peered":"2024-03-04T12:11:53.882147+0000","last_unstale":"2024-03-04T12:12:00.771668+0000","last_undegraded":"2024-03-04T12:12:00.771668+0000","last_fullsized":"2024-03-04T12:12:00.771668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771019+0000","last_change":"2024-03-04T12:11:58.946384+0000","last_active":"2024-03-04T12:12:00.771019+0000","last_peered":"2024-03-04T12:12:00.771019+0000","last_clean":"2024-03-04T12:12:00.771019+0000","last_became_active":"2024-03-04T12:11:58.945989+0000","last_became_peered":"2024-03-04T12:11:58.945989+0000","last_unstale":"2024-03-04T12:12:00.771019+0000","last_undegraded":"2024-03-04T12:12:00.771019+0000","last_fullsized":"2024-03-04T12:12:00.771019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772496+0000","last_change":"2024-03-04T12:11:54.903131+0000","last_active":"2024-03-04T12:12:00.772496+0000","last_peered":"2024-03-04T12:12:00.772496+0000","last_clean":"2024-03-04T12:12:00.772496+0000","last_became_active":"2024-03-04T12:11:54.902729+0000","last_became_peered":"2024-03-04T12:11:54.902729+0000","last_unstale":"2024-03-04T12:12:00.772496+0000","last_undegraded":"2024-03-04T12:12:00.772496+0000","last_fullsized":"2024-03-04T12:12:00.772496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:43.148028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921116+0000","last_change":"2024-03-04T12:11:53.884623+0000","last_active":"2024-03-04T12:11:59.921116+0000","last_peered":"2024-03-04T12:11:59.921116+0000","last_clean":"2024-03-04T12:11:59.921116+0000","last_became_active":"2024-03-04T12:11:53.884222+0000","last_became_peered":"2024-03-04T12:11:53.884222+0000","last_unstale":"2024-03-04T12:11:59.921116+0000","last_undegraded":"2024-03-04T12:11:59.921116+0000","last_fullsized":"2024-03-04T12:11:59.921116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771150+0000","last_change":"2024-03-04T12:11:58.935385+0000","last_active":"2024-03-04T12:12:00.771150+0000","last_peered":"2024-03-04T12:12:00.771150+0000","last_clean":"2024-03-04T12:12:00.771150+0000","last_became_active":"2024-03-04T12:11:58.934991+0000","last_became_peered":"2024-03-04T12:11:58.934991+0000","last_unstale":"2024-03-04T12:12:00.771150+0000","last_undegraded":"2024-03-04T12:12:00.771150+0000","last_fullsized":"2024-03-04T12:12:00.771150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:09.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773010+0000","last_change":"2024-03-04T12:11:54.901559+0000","last_active":"2024-03-04T12:12:00.773010+0000","last_peered":"2024-03-04T12:12:00.773010+0000","last_clean":"2024-03-04T12:12:00.773010+0000","last_became_active":"2024-03-04T12:11:54.901215+0000","last_became_peered":"2024-03-04T12:11:54.901215+0000","last_unstale":"2024-03-04T12:12:00.773010+0000","last_undegraded":"2024-03-04T12:12:00.773010+0000","last_fullsized":"2024-03-04T12:12:00.773010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772226+0000","last_change":"2024-03-04T12:11:53.884589+0000","last_active":"2024-03-04T12:12:00.772226+0000","last_peered":"2024-03-04T12:12:00.772226+0000","last_clean":"2024-03-04T12:12:00.772226+0000","last_became_active":"2024-03-04T12:11:53.884279+0000","last_became_peered":"2024-03-04T12:11:53.884279+0000","last_unstale":"2024-03-04T12:12:00.772226+0000","last_undegraded":"2024-03-04T12:12:00.772226+0000","last_fullsized":"2024-03-04T12:12:00.772226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:54.931530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771590+0000","last_change":"2024-03-04T12:11:58.945665+0000","last_active":"2024-03-04T12:12:00.771590+0000","last_peered":"2024-03-04T12:12:00.771590+0000","last_clean":"2024-03-04T12:12:00.771590+0000","last_became_active":"2024-03-04T12:11:58.945297+0000","last_became_peered":"2024-03-04T12:11:58.945297+0000","last_unstale":"2024-03-04T12:12:00.771590+0000","last_undegraded":"2024-03-04T12:12:00.771590+0000","last_fullsized":"2024-03-04T12:12:00.771590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:11.565262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771183+0000","last_change":"2024-03-04T12:11:54.902251+0000","last_active":"2024-03-04T12:12:00.771183+0000","last_peered":"2024-03-04T12:12:00.771183+0000","last_clean":"2024-03-04T12:12:00.771183+0000","last_became_active":"2024-03-04T12:11:54.901942+0000","last_became_peered":"2024-03-04T12:11:54.901942+0000","last_unstale":"2024-03-04T12:12:00.771183+0000","last_undegraded":"2024-03-04T12:12:00.771183+0000","last_fullsized":"2024-03-04T12:12:00.771183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920811+0000","last_change":"2024-03-04T12:11:53.883599+0000","last_active":"2024-03-04T12:11:59.920811+0000","last_peered":"2024-03-04T12:11:59.920811+0000","last_clean":"2024-03-04T12:11:59.920811+0000","last_became_active":"2024-03-04T12:11:53.883317+0000","last_became_peered":"2024-03-04T12:11:53.883317+0000","last_unstale":"2024-03-04T12:11:59.920811+0000","last_undegraded":"2024-03-04T12:11:59.920811+0000","last_fullsized":"2024-03-04T12:11:59.920811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771562+0000","last_change":"2024-03-04T12:11:58.936436+0000","last_active":"2024-03-04T12:12:00.771562+0000","last_peered":"2024-03-04T12:12:00.771562+0000","last_clean":"2024-03-04T12:12:00.771562+0000","last_became_active":"2024-03-04T12:11:58.935593+0000","last_became_peered":"2024-03-04T12:11:58.935593+0000","last_unstale":"2024-03-04T12:12:00.771562+0000","last_undegraded":"2024-03-04T12:12:00.771562+0000","last_fullsized":"2024-03-04T12:12:00.771562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920930+0000","last_change":"2024-03-04T12:11:54.908033+0000","last_active":"2024-03-04T12:11:59.920930+0000","last_peered":"2024-03-04T12:11:59.920930+0000","last_clean":"2024-03-04T12:11:59.920930+0000","last_became_active":"2024-03-04T12:11:54.907784+0000","last_became_peered":"2024-03-04T12:11:54.907784+0000","last_unstale":"2024-03-04T12:11:59.920930+0000","last_undegraded":"2024-03-04T12:11:59.920930+0000","last_fullsized":"2024-03-04T12:11:59.920930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921086+0000","last_change":"2024-03-04T12:11:53.889166+0000","last_active":"2024-03-04T12:11:59.921086+0000","last_peered":"2024-03-04T12:11:59.921086+0000","last_clean":"2024-03-04T12:11:59.921086+0000","last_became_active":"2024-03-04T12:11:53.888906+0000","last_became_peered":"2024-03-04T12:11:53.888906+0000","last_unstale":"2024-03-04T12:11:59.921086+0000","last_undegraded":"2024-03-04T12:11:59.921086+0000","last_fullsized":"2024-03-04T12:11:59.921086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921204+0000","last_change":"2024-03-04T12:11:58.944078+0000","last_active":"2024-03-04T12:11:59.921204+0000","last_peered":"2024-03-04T12:11:59.921204+0000","last_clean":"2024-03-04T12:11:59.921204+0000","last_became_active":"2024-03-04T12:11:58.942753+0000","last_became_peered":"2024-03-04T12:11:58.942753+0000","last_unstale":"2024-03-04T12:11:59.921204+0000","last_undegraded":"2024-03-04T12:11:59.921204+0000","last_fullsized":"2024-03-04T12:11:59.921204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:35.014220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772005+0000","last_change":"2024-03-04T12:11:54.902835+0000","last_active":"2024-03-04T12:12:00.772005+0000","last_peered":"2024-03-04T12:12:00.772005+0000","last_clean":"2024-03-04T12:12:00.772005+0000","last_became_active":"2024-03-04T12:11:54.902550+0000","last_became_peered":"2024-03-04T12:11:54.902550+0000","last_unstale":"2024-03-04T12:12:00.772005+0000","last_undegraded":"2024-03-04T12:12:00.772005+0000","last_fullsized":"2024-03-04T12:12:00.772005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:53.885828+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:53.885526+0000","last_became_peered":"2024-03-04T12:11:53.885526+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920921+0000","last_change":"2024-03-04T12:11:58.943710+0000","last_active":"2024-03-04T12:11:59.920921+0000","last_peered":"2024-03-04T12:11:59.920921+0000","last_clean":"2024-03-04T12:11:59.920921+0000","last_became_active":"2024-03-04T12:11:58.943297+0000","last_became_peered":"2024-03-04T12:11:58.943297+0000","last_unstale":"2024-03-04T12:11:59.920921+0000","last_undegraded":"2024-03-04T12:11:59.920921+0000","last_fullsized":"2024-03-04T12:11:59.920921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:56.606927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770672+0000","last_change":"2024-03-04T12:11:54.900415+0000","last_active":"2024-03-04T12:12:00.770672+0000","last_peered":"2024-03-04T12:12:00.770672+0000","last_clean":"2024-03-04T12:12:00.770672+0000","last_became_active":"2024-03-04T12:11:54.900101+0000","last_became_peered":"2024-03-04T12:11:54.900101+0000","last_unstale":"2024-03-04T12:12:00.770672+0000","last_undegraded":"2024-03-04T12:12:00.770672+0000","last_fullsized":"2024-03-04T12:12:00.770672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771029+0000","last_change":"2024-03-04T12:11:53.888493+0000","last_active":"2024-03-04T12:12:00.771029+0000","last_peered":"2024-03-04T12:12:00.771029+0000","last_clean":"2024-03-04T12:12:00.771029+0000","last_became_active":"2024-03-04T12:11:53.888223+0000","last_became_peered":"2024-03-04T12:11:53.888223+0000","last_unstale":"2024-03-04T12:12:00.771029+0000","last_undegraded":"2024-03-04T12:12:00.771029+0000","last_fullsized":"2024-03-04T12:12:00.771029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:40.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772694+0000","last_change":"2024-03-04T12:11:58.934500+0000","last_active":"2024-03-04T12:12:00.772694+0000","last_peered":"2024-03-04T12:12:00.772694+0000","last_clean":"2024-03-04T12:12:00.772694+0000","last_became_active":"2024-03-04T12:11:58.933910+0000","last_became_peered":"2024-03-04T12:11:58.933910+0000","last_unstale":"2024-03-04T12:12:00.772694+0000","last_undegraded":"2024-03-04T12:12:00.772694+0000","last_fullsized":"2024-03-04T12:12:00.772694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771718+0000","last_change":"2024-03-04T12:11:54.906835+0000","last_active":"2024-03-04T12:12:00.771718+0000","last_peered":"2024-03-04T12:12:00.771718+0000","last_clean":"2024-03-04T12:12:00.771718+0000","last_became_active":"2024-03-04T12:11:54.904477+0000","last_became_peered":"2024-03-04T12:11:54.904477+0000","last_unstale":"2024-03-04T12:12:00.771718+0000","last_undegraded":"2024-03-04T12:12:00.771718+0000","last_fullsized":"2024-03-04T12:12:00.771718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:04.123314+0000","last_change":"2024-03-04T12:11:53.889752+0000","last_active":"2024-03-04T12:12:04.123314+0000","last_peered":"2024-03-04T12:12:04.123314+0000","last_clean":"2024-03-04T12:12:04.123314+0000","last_became_active":"2024-03-04T12:11:53.889469+0000","last_became_peered":"2024-03-04T12:11:53.889469+0000","last_unstale":"2024-03-04T12:12:04.123314+0000","last_undegraded":"2024-03-04T12:12:04.123314+0000","last_fullsized":"2024-03-04T12:12:04.123314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771293+0000","last_change":"2024-03-04T12:11:58.941223+0000","last_active":"2024-03-04T12:12:00.771293+0000","last_peered":"2024-03-04T12:12:00.771293+0000","last_clean":"2024-03-04T12:12:00.771293+0000","last_became_active":"2024-03-04T12:11:58.940845+0000","last_became_peered":"2024-03-04T12:11:58.940845+0000","last_unstale":"2024-03-04T12:12:00.771293+0000","last_undegraded":"2024-03-04T12:12:00.771293+0000","last_fullsized":"2024-03-04T12:12:00.771293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772686+0000","last_change":"2024-03-04T12:11:54.902174+0000","last_active":"2024-03-04T12:12:00.772686+0000","last_peered":"2024-03-04T12:12:00.772686+0000","last_clean":"2024-03-04T12:12:00.772686+0000","last_became_active":"2024-03-04T12:11:54.901884+0000","last_became_peered":"2024-03-04T12:11:54.901884+0000","last_unstale":"2024-03-04T12:12:00.772686+0000","last_undegraded":"2024-03-04T12:12:00.772686+0000","last_fullsized":"2024-03-04T12:12:00.772686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771560+0000","last_change":"2024-03-04T12:11:53.892494+0000","last_active":"2024-03-04T12:12:00.771560+0000","last_peered":"2024-03-04T12:12:00.771560+0000","last_clean":"2024-03-04T12:12:00.771560+0000","last_became_active":"2024-03-04T12:11:53.892170+0000","last_became_peered":"2024-03-04T12:11:53.892170+0000","last_unstale":"2024-03-04T12:12:00.771560+0000","last_undegraded":"2024-03-04T12:12:00.771560+0000","last_fullsized":"2024-03-04T12:12:00.771560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773086+0000","last_change":"2024-03-04T12:11:58.945966+0000","last_active":"2024-03-04T12:12:00.773086+0000","last_peered":"2024-03-04T12:12:00.773086+0000","last_clean":"2024-03-04T12:12:00.773086+0000","last_became_active":"2024-03-04T12:11:58.945545+0000","last_became_peered":"2024-03-04T12:11:58.945545+0000","last_unstale":"2024-03-04T12:12:00.773086+0000","last_undegraded":"2024-03-04T12:12:00.773086+0000","last_fullsized":"2024-03-04T12:12:00.773086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771604+0000","last_change":"2024-03-04T12:11:54.900788+0000","last_active":"2024-03-04T12:12:00.771604+0000","last_peered":"2024-03-04T12:12:00.771604+0000","last_clean":"2024-03-04T12:12:00.771604+0000","last_became_active":"2024-03-04T12:11:54.900523+0000","last_became_peered":"2024-03-04T12:11:54.900523+0000","last_unstale":"2024-03-04T12:12:00.771604+0000","last_undegraded":"2024-03-04T12:12:00.771604+0000","last_fullsized":"2024-03-04T12:12:00.771604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773050+0000","last_change":"2024-03-04T12:11:53.887458+0000","last_active":"2024-03-04T12:12:00.773050+0000","last_peered":"2024-03-04T12:12:00.773050+0000","last_clean":"2024-03-04T12:12:00.773050+0000","last_became_active":"2024-03-04T12:11:53.887205+0000","last_became_peered":"2024-03-04T12:11:53.887205+0000","last_unstale":"2024-03-04T12:12:00.773050+0000","last_undegraded":"2024-03-04T12:12:00.773050+0000","last_fullsized":"2024-03-04T12:12:00.773050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920213+0000","last_change":"2024-03-04T12:11:58.944437+0000","last_active":"2024-03-04T12:11:59.920213+0000","last_peered":"2024-03-04T12:11:59.920213+0000","last_clean":"2024-03-04T12:11:59.920213+0000","last_became_active":"2024-03-04T12:11:58.944027+0000","last_became_peered":"2024-03-04T12:11:58.944027+0000","last_unstale":"2024-03-04T12:11:59.920213+0000","last_undegraded":"2024-03-04T12:11:59.920213+0000","last_fullsized":"2024-03-04T12:11:59.920213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:09:47.849893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773158+0000","last_change":"2024-03-04T12:11:54.904517+0000","last_active":"2024-03-04T12:12:00.773158+0000","last_peered":"2024-03-04T12:12:00.773158+0000","last_clean":"2024-03-04T12:12:00.773158+0000","last_became_active":"2024-03-04T12:11:54.904245+0000","last_became_peered":"2024-03-04T12:11:54.904245+0000","last_unstale":"2024-03-04T12:12:00.773158+0000","last_undegraded":"2024-03-04T12:12:00.773158+0000","last_fullsized":"2024-03-04T12:12:00.773158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921498+0000","last_change":"2024-03-04T12:11:53.888433+0000","last_active":"2024-03-04T12:11:59.921498+0000","last_peered":"2024-03-04T12:11:59.921498+0000","last_clean":"2024-03-04T12:11:59.921498+0000","last_became_active":"2024-03-04T12:11:53.888164+0000","last_became_peered":"2024-03-04T12:11:53.888164+0000","last_unstale":"2024-03-04T12:11:59.921498+0000","last_undegraded":"2024-03-04T12:11:59.921498+0000","last_fullsized":"2024-03-04T12:11:59.921498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920282+0000","last_change":"2024-03-04T12:11:58.937226+0000","last_active":"2024-03-04T12:11:59.920282+0000","last_peered":"2024-03-04T12:11:59.920282+0000","last_clean":"2024-03-04T12:11:59.920282+0000","last_became_active":"2024-03-04T12:11:58.936831+0000","last_became_peered":"2024-03-04T12:11:58.936831+0000","last_unstale":"2024-03-04T12:11:59.920282+0000","last_undegraded":"2024-03-04T12:11:59.920282+0000","last_fullsized":"2024-03-04T12:11:59.920282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:02.110563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770587+0000","last_change":"2024-03-04T12:11:58.946973+0000","last_active":"2024-03-04T12:12:00.770587+0000","last_peered":"2024-03-04T12:12:00.770587+0000","last_clean":"2024-03-04T12:12:00.770587+0000","last_became_active":"2024-03-04T12:11:58.946581+0000","last_became_peered":"2024-03-04T12:11:58.946581+0000","last_unstale":"2024-03-04T12:12:00.770587+0000","last_undegraded":"2024-03-04T12:12:00.770587+0000","last_fullsized":"2024-03-04T12:12:00.770587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770784+0000","last_change":"2024-03-04T12:11:54.900431+0000","last_active":"2024-03-04T12:12:00.770784+0000","last_peered":"2024-03-04T12:12:00.770784+0000","last_clean":"2024-03-04T12:12:00.770784+0000","last_became_active":"2024-03-04T12:11:54.900166+0000","last_became_peered":"2024-03-04T12:11:54.900166+0000","last_unstale":"2024-03-04T12:12:00.770784+0000","last_undegraded":"2024-03-04T12:12:00.770784+0000","last_fullsized":"2024-03-04T12:12:00.770784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:53.879757+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:53.878758+0000","last_became_peered":"2024-03-04T12:11:53.878758+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921468+0000","last_change":"2024-03-04T12:11:58.945414+0000","last_active":"2024-03-04T12:11:59.921468+0000","last_peered":"2024-03-04T12:11:59.921468+0000","last_clean":"2024-03-04T12:11:59.921468+0000","last_became_active":"2024-03-04T12:11:58.945031+0000","last_became_peered":"2024-03-04T12:11:58.945031+0000","last_unstale":"2024-03-04T12:11:59.921468+0000","last_undegraded":"2024-03-04T12:11:59.921468+0000","last_fullsized":"2024-03-04T12:11:59.921468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920217+0000","last_change":"2024-03-04T12:11:54.896991+0000","last_active":"2024-03-04T12:11:59.920217+0000","last_peered":"2024-03-04T12:11:59.920217+0000","last_clean":"2024-03-04T12:11:59.920217+0000","last_became_active":"2024-03-04T12:11:54.896725+0000","last_became_peered":"2024-03-04T12:11:54.896725+0000","last_unstale":"2024-03-04T12:11:59.920217+0000","last_undegraded":"2024-03-04T12:11:59.920217+0000","last_fullsized":"2024-03-04T12:11:59.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920530+0000","last_change":"2024-03-04T12:11:53.884328+0000","last_active":"2024-03-04T12:11:59.920530+0000","last_peered":"2024-03-04T12:11:59.920530+0000","last_clean":"2024-03-04T12:11:59.920530+0000","last_became_active":"2024-03-04T12:11:53.882865+0000","last_became_peered":"2024-03-04T12:11:53.882865+0000","last_unstale":"2024-03-04T12:11:59.920530+0000","last_undegraded":"2024-03-04T12:11:59.920530+0000","last_fullsized":"2024-03-04T12:11:59.920530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921283+0000","last_change":"2024-03-04T12:11:53.881932+0000","last_active":"2024-03-04T12:11:59.921283+0000","last_peered":"2024-03-04T12:11:59.921283+0000","last_clean":"2024-03-04T12:11:59.921283+0000","last_became_active":"2024-03-04T12:11:53.880843+0000","last_became_peered":"2024-03-04T12:11:53.880843+0000","last_unstale":"2024-03-04T12:11:59.921283+0000","last_undegraded":"2024-03-04T12:11:59.921283+0000","last_fullsized":"2024-03-04T12:11:59.921283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770941+0000","last_change":"2024-03-04T12:11:54.892168+0000","last_active":"2024-03-04T12:12:00.770941+0000","last_peered":"2024-03-04T12:12:00.770941+0000","last_clean":"2024-03-04T12:12:00.770941+0000","last_became_active":"2024-03-04T12:11:54.891886+0000","last_became_peered":"2024-03-04T12:11:54.891886+0000","last_unstale":"2024-03-04T12:12:00.770941+0000","last_undegraded":"2024-03-04T12:12:00.770941+0000","last_fullsized":"2024-03-04T12:12:00.770941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920785+0000","last_change":"2024-03-04T12:11:58.946859+0000","last_active":"2024-03-04T12:11:59.920785+0000","last_peered":"2024-03-04T12:11:59.920785+0000","last_clean":"2024-03-04T12:11:59.920785+0000","last_became_active":"2024-03-04T12:11:58.946495+0000","last_became_peered":"2024-03-04T12:11:58.946495+0000","last_unstale":"2024-03-04T12:11:59.920785+0000","last_undegraded":"2024-03-04T12:11:59.920785+0000","last_fullsized":"2024-03-04T12:11:59.920785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:25.320885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920580+0000","last_change":"2024-03-04T12:11:53.889151+0000","last_active":"2024-03-04T12:11:59.920580+0000","last_peered":"2024-03-04T12:11:59.920580+0000","last_clean":"2024-03-04T12:11:59.920580+0000","last_became_active":"2024-03-04T12:11:53.888882+0000","last_became_peered":"2024-03-04T12:11:53.888882+0000","last_unstale":"2024-03-04T12:11:59.920580+0000","last_undegraded":"2024-03-04T12:11:59.920580+0000","last_fullsized":"2024-03-04T12:11:59.920580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772788+0000","last_change":"2024-03-04T12:11:54.901800+0000","last_active":"2024-03-04T12:12:00.772788+0000","last_peered":"2024-03-04T12:12:00.772788+0000","last_clean":"2024-03-04T12:12:00.772788+0000","last_became_active":"2024-03-04T12:11:54.901502+0000","last_became_peered":"2024-03-04T12:11:54.901502+0000","last_unstale":"2024-03-04T12:12:00.772788+0000","last_undegraded":"2024-03-04T12:12:00.772788+0000","last_fullsized":"2024-03-04T12:12:00.772788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:36.238592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921109+0000","last_change":"2024-03-04T12:11:58.943607+0000","last_active":"2024-03-04T12:11:59.921109+0000","last_peered":"2024-03-04T12:11:59.921109+0000","last_clean":"2024-03-04T12:11:59.921109+0000","last_became_active":"2024-03-04T12:11:58.942837+0000","last_became_peered":"2024-03-04T12:11:58.942837+0000","last_unstale":"2024-03-04T12:11:59.921109+0000","last_undegraded":"2024-03-04T12:11:59.921109+0000","last_fullsized":"2024-03-04T12:11:59.921109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:59.670150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920801+0000","last_change":"2024-03-04T12:11:54.908034+0000","last_active":"2024-03-04T12:11:59.920801+0000","last_peered":"2024-03-04T12:11:59.920801+0000","last_clean":"2024-03-04T12:11:59.920801+0000","last_became_active":"2024-03-04T12:11:54.907761+0000","last_became_peered":"2024-03-04T12:11:54.907761+0000","last_unstale":"2024-03-04T12:11:59.920801+0000","last_undegraded":"2024-03-04T12:11:59.920801+0000","last_fullsized":"2024-03-04T12:11:59.920801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921098+0000","last_change":"2024-03-04T12:11:54.904118+0000","last_active":"2024-03-04T12:11:59.921098+0000","last_peered":"2024-03-04T12:11:59.921098+0000","last_clean":"2024-03-04T12:11:59.921098+0000","last_became_active":"2024-03-04T12:11:54.903747+0000","last_became_peered":"2024-03-04T12:11:54.903747+0000","last_unstale":"2024-03-04T12:11:59.921098+0000","last_undegraded":"2024-03-04T12:11:59.921098+0000","last_fullsized":"2024-03-04T12:11:59.921098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771638+0000","last_change":"2024-03-04T12:11:54.900197+0000","last_active":"2024-03-04T12:12:00.771638+0000","last_peered":"2024-03-04T12:12:00.771638+0000","last_clean":"2024-03-04T12:12:00.771638+0000","last_became_active":"2024-03-04T12:11:54.899939+0000","last_became_peered":"2024-03-04T12:11:54.899939+0000","last_unstale":"2024-03-04T12:12:00.771638+0000","last_undegraded":"2024-03-04T12:12:00.771638+0000","last_fullsized":"2024-03-04T12:12:00.771638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770848+0000","last_change":"2024-03-04T12:11:54.896614+0000","last_active":"2024-03-04T12:12:00.770848+0000","last_peered":"2024-03-04T12:12:00.770848+0000","last_clean":"2024-03-04T12:12:00.770848+0000","last_became_active":"2024-03-04T12:11:54.896312+0000","last_became_peered":"2024-03-04T12:11:54.896312+0000","last_unstale":"2024-03-04T12:12:00.770848+0000","last_undegraded":"2024-03-04T12:12:00.770848+0000","last_fullsized":"2024-03-04T12:12:00.770848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920990+0000","last_change":"2024-03-04T12:11:54.898597+0000","last_active":"2024-03-04T12:11:59.920990+0000","last_peered":"2024-03-04T12:11:59.920990+0000","last_clean":"2024-03-04T12:11:59.920990+0000","last_became_active":"2024-03-04T12:11:54.898339+0000","last_became_peered":"2024-03-04T12:11:54.898339+0000","last_unstale":"2024-03-04T12:11:59.920990+0000","last_undegraded":"2024-03-04T12:11:59.920990+0000","last_fullsized":"2024-03-04T12:11:59.920990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771385+0000","last_change":"2024-03-04T12:11:54.902225+0000","last_active":"2024-03-04T12:12:00.771385+0000","last_peered":"2024-03-04T12:12:00.771385+0000","last_clean":"2024-03-04T12:12:00.771385+0000","last_became_active":"2024-03-04T12:11:54.901968+0000","last_became_peered":"2024-03-04T12:11:54.901968+0000","last_unstale":"2024-03-04T12:12:00.771385+0000","last_undegraded":"2024-03-04T12:12:00.771385+0000","last_fullsized":"2024-03-04T12:12:00.771385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921621+0000","last_change":"2024-03-04T12:11:54.911542+0000","last_active":"2024-03-04T12:11:59.921621+0000","last_peered":"2024-03-04T12:11:59.921621+0000","last_clean":"2024-03-04T12:11:59.921621+0000","last_became_active":"2024-03-04T12:11:54.911032+0000","last_became_peered":"2024-03-04T12:11:54.911032+0000","last_unstale":"2024-03-04T12:11:59.921621+0000","last_undegraded":"2024-03-04T12:11:59.921621+0000","last_fullsized":"2024-03-04T12:11:59.921621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921271+0000","last_change":"2024-03-04T12:11:54.897959+0000","last_active":"2024-03-04T12:11:59.921271+0000","last_peered":"2024-03-04T12:11:59.921271+0000","last_clean":"2024-03-04T12:11:59.921271+0000","last_became_active":"2024-03-04T12:11:54.897665+0000","last_became_peered":"2024-03-04T12:11:54.897665+0000","last_unstale":"2024-03-04T12:11:59.921271+0000","last_undegraded":"2024-03-04T12:11:59.921271+0000","last_fullsized":"2024-03-04T12:11:59.921271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:54.899561+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:54.899289+0000","last_became_peered":"2024-03-04T12:11:54.899289+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920409+0000","last_change":"2024-03-04T12:11:54.897860+0000","last_active":"2024-03-04T12:11:59.920409+0000","last_peered":"2024-03-04T12:11:59.920409+0000","last_clean":"2024-03-04T12:11:59.920409+0000","last_became_active":"2024-03-04T12:11:54.897585+0000","last_became_peered":"2024-03-04T12:11:54.897585+0000","last_unstale":"2024-03-04T12:11:59.920409+0000","last_undegraded":"2024-03-04T12:11:59.920409+0000","last_fullsized":"2024-03-04T12:11:59.920409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771542+0000","last_change":"2024-03-04T12:11:54.904095+0000","last_active":"2024-03-04T12:12:00.771542+0000","last_peered":"2024-03-04T12:12:00.771542+0000","last_clean":"2024-03-04T12:12:00.771542+0000","last_became_active":"2024-03-04T12:11:54.903527+0000","last_became_peered":"2024-03-04T12:11:54.903527+0000","last_unstale":"2024-03-04T12:12:00.771542+0000","last_undegraded":"2024-03-04T12:12:00.771542+0000","last_fullsized":"2024-03-04T12:12:00.771542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770740+0000","last_change":"2024-03-04T12:11:54.904758+0000","last_active":"2024-03-04T12:12:00.770740+0000","last_peered":"2024-03-04T12:12:00.770740+0000","last_clean":"2024-03-04T12:12:00.770740+0000","last_became_active":"2024-03-04T12:11:54.904480+0000","last_became_peered":"2024-03-04T12:11:54.904480+0000","last_unstale":"2024-03-04T12:12:00.770740+0000","last_undegraded":"2024-03-04T12:12:00.770740+0000","last_fullsized":"2024-03-04T12:12:00.770740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921331+0000","last_change":"2024-03-04T12:11:54.901144+0000","last_active":"2024-03-04T12:11:59.921331+0000","last_peered":"2024-03-04T12:11:59.921331+0000","last_clean":"2024-03-04T12:11:59.921331+0000","last_became_active":"2024-03-04T12:11:54.900739+0000","last_became_peered":"2024-03-04T12:11:54.900739+0000","last_unstale":"2024-03-04T12:11:59.921331+0000","last_undegraded":"2024-03-04T12:11:59.921331+0000","last_fullsized":"2024-03-04T12:11:59.921331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771914+0000","last_change":"2024-03-04T12:11:54.905995+0000","last_active":"2024-03-04T12:12:00.771914+0000","last_peered":"2024-03-04T12:12:00.771914+0000","last_clean":"2024-03-04T12:12:00.771914+0000","last_became_active":"2024-03-04T12:11:54.904878+0000","last_became_peered":"2024-03-04T12:11:54.904878+0000","last_unstale":"2024-03-04T12:12:00.771914+0000","last_undegraded":"2024-03-04T12:12:00.771914+0000","last_fullsized":"2024-03-04T12:12:00.771914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921040+0000","last_change":"2024-03-04T12:11:54.904652+0000","last_active":"2024-03-04T12:11:59.921040+0000","last_peered":"2024-03-04T12:11:59.921040+0000","last_clean":"2024-03-04T12:11:59.921040+0000","last_became_active":"2024-03-04T12:11:54.904348+0000","last_became_peered":"2024-03-04T12:11:54.904348+0000","last_unstale":"2024-03-04T12:11:59.921040+0000","last_undegraded":"2024-03-04T12:11:59.921040+0000","last_fullsized":"2024-03-04T12:11:59.921040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:05.687532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772261+0000","last_change":"2024-03-04T12:11:54.905800+0000","last_active":"2024-03-04T12:12:00.772261+0000","last_peered":"2024-03-04T12:12:00.772261+0000","last_clean":"2024-03-04T12:12:00.772261+0000","last_became_active":"2024-03-04T12:11:54.905550+0000","last_became_peered":"2024-03-04T12:11:54.905550+0000","last_unstale":"2024-03-04T12:12:00.772261+0000","last_undegraded":"2024-03-04T12:12:00.772261+0000","last_fullsized":"2024-03-04T12:12:00.772261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771115+0000","last_change":"2024-03-04T12:11:54.909971+0000","last_active":"2024-03-04T12:12:00.771115+0000","last_peered":"2024-03-04T12:12:00.771115+0000","last_clean":"2024-03-04T12:12:00.771115+0000","last_became_active":"2024-03-04T12:11:54.909694+0000","last_became_peered":"2024-03-04T12:11:54.909694+0000","last_unstale":"2024-03-04T12:12:00.771115+0000","last_undegraded":"2024-03-04T12:12:00.771115+0000","last_fullsized":"2024-03-04T12:12:00.771115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770786+0000","last_change":"2024-03-04T12:11:54.901482+0000","last_active":"2024-03-04T12:12:00.770786+0000","last_peered":"2024-03-04T12:12:00.770786+0000","last_clean":"2024-03-04T12:12:00.770786+0000","last_became_active":"2024-03-04T12:11:54.901185+0000","last_became_peered":"2024-03-04T12:11:54.901185+0000","last_unstale":"2024-03-04T12:12:00.770786+0000","last_undegraded":"2024-03-04T12:12:00.770786+0000","last_fullsized":"2024-03-04T12:12:00.770786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770638+0000","last_change":"2024-03-04T12:11:54.902348+0000","last_active":"2024-03-04T12:12:00.770638+0000","last_peered":"2024-03-04T12:12:00.770638+0000","last_clean":"2024-03-04T12:12:00.770638+0000","last_became_active":"2024-03-04T12:11:54.902087+0000","last_became_peered":"2024-03-04T12:11:54.902087+0000","last_unstale":"2024-03-04T12:12:00.770638+0000","last_undegraded":"2024-03-04T12:12:00.770638+0000","last_fullsized":"2024-03-04T12:12:00.770638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:04:00.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920436+0000","last_change":"2024-03-04T12:11:54.904336+0000","last_active":"2024-03-04T12:11:59.920436+0000","last_peered":"2024-03-04T12:11:59.920436+0000","last_clean":"2024-03-04T12:11:59.920436+0000","last_became_active":"2024-03-04T12:11:54.904049+0000","last_became_peered":"2024-03-04T12:11:54.904049+0000","last_unstale":"2024-03-04T12:11:59.920436+0000","last_undegraded":"2024-03-04T12:11:59.920436+0000","last_fullsized":"2024-03-04T12:11:59.920436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772596+0000","last_change":"2024-03-04T12:11:54.907818+0000","last_active":"2024-03-04T12:12:00.772596+0000","last_peered":"2024-03-04T12:12:00.772596+0000","last_clean":"2024-03-04T12:12:00.772596+0000","last_became_active":"2024-03-04T12:11:54.907443+0000","last_became_peered":"2024-03-04T12:11:54.907443+0000","last_unstale":"2024-03-04T12:12:00.772596+0000","last_undegraded":"2024-03-04T12:12:00.772596+0000","last_fullsized":"2024-03-04T12:12:00.772596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:26.417645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921105+0000","last_change":"2024-03-04T12:11:54.906714+0000","last_active":"2024-03-04T12:11:59.921105+0000","last_peered":"2024-03-04T12:11:59.921105+0000","last_clean":"2024-03-04T12:11:59.921105+0000","last_became_active":"2024-03-04T12:11:54.906367+0000","last_became_peered":"2024-03-04T12:11:54.906367+0000","last_unstale":"2024-03-04T12:11:59.921105+0000","last_undegraded":"2024-03-04T12:11:59.921105+0000","last_fullsized":"2024-03-04T12:11:59.921105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921624+0000","last_change":"2024-03-04T12:11:54.908667+0000","last_active":"2024-03-04T12:11:59.921624+0000","last_peered":"2024-03-04T12:11:59.921624+0000","last_clean":"2024-03-04T12:11:59.921624+0000","last_became_active":"2024-03-04T12:11:54.908377+0000","last_became_peered":"2024-03-04T12:11:54.908377+0000","last_unstale":"2024-03-04T12:11:59.921624+0000","last_undegraded":"2024-03-04T12:11:59.921624+0000","last_fullsized":"2024-03-04T12:11:59.921624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772398+0000","last_change":"2024-03-04T12:11:54.907669+0000","last_active":"2024-03-04T12:12:00.772398+0000","last_peered":"2024-03-04T12:12:00.772398+0000","last_clean":"2024-03-04T12:12:00.772398+0000","last_became_active":"2024-03-04T12:11:54.905673+0000","last_became_peered":"2024-03-04T12:11:54.905673+0000","last_unstale":"2024-03-04T12:12:00.772398+0000","last_undegraded":"2024-03-04T12:12:00.772398+0000","last_fullsized":"2024-03-04T12:12:00.772398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:43.980065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920103+0000","last_change":"2024-03-04T12:11:54.904733+0000","last_active":"2024-03-04T12:11:59.920103+0000","last_peered":"2024-03-04T12:11:59.920103+0000","last_clean":"2024-03-04T12:11:59.920103+0000","last_became_active":"2024-03-04T12:11:54.903680+0000","last_became_peered":"2024-03-04T12:11:54.903680+0000","last_unstale":"2024-03-04T12:11:59.920103+0000","last_undegraded":"2024-03-04T12:11:59.920103+0000","last_fullsized":"2024-03-04T12:11:59.920103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771078+0000","last_change":"2024-03-04T12:11:54.902367+0000","last_active":"2024-03-04T12:12:00.771078+0000","last_peered":"2024-03-04T12:12:00.771078+0000","last_clean":"2024-03-04T12:12:00.771078+0000","last_became_active":"2024-03-04T12:11:54.902075+0000","last_became_peered":"2024-03-04T12:11:54.902075+0000","last_unstale":"2024-03-04T12:12:00.771078+0000","last_undegraded":"2024-03-04T12:12:00.771078+0000","last_fullsized":"2024-03-04T12:12:00.771078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771398+0000","last_change":"2024-03-04T12:11:54.899399+0000","last_active":"2024-03-04T12:12:00.771398+0000","last_peered":"2024-03-04T12:12:00.771398+0000","last_clean":"2024-03-04T12:12:00.771398+0000","last_became_active":"2024-03-04T12:11:54.899084+0000","last_became_peered":"2024-03-04T12:11:54.899084+0000","last_unstale":"2024-03-04T12:12:00.771398+0000","last_undegraded":"2024-03-04T12:12:00.771398+0000","last_fullsized":"2024-03-04T12:12:00.771398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772929+0000","last_change":"2024-03-04T12:11:54.903214+0000","last_active":"2024-03-04T12:12:00.772929+0000","last_peered":"2024-03-04T12:12:00.772929+0000","last_clean":"2024-03-04T12:12:00.772929+0000","last_became_active":"2024-03-04T12:11:54.901705+0000","last_became_peered":"2024-03-04T12:11:54.901705+0000","last_unstale":"2024-03-04T12:12:00.772929+0000","last_undegraded":"2024-03-04T12:12:00.772929+0000","last_fullsized":"2024-03-04T12:12:00.772929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771893+0000","last_change":"2024-03-04T12:11:54.910691+0000","last_active":"2024-03-04T12:12:00.771893+0000","last_peered":"2024-03-04T12:12:00.771893+0000","last_clean":"2024-03-04T12:12:00.771893+0000","last_became_active":"2024-03-04T12:11:54.910198+0000","last_became_peered":"2024-03-04T12:11:54.910198+0000","last_unstale":"2024-03-04T12:12:00.771893+0000","last_undegraded":"2024-03-04T12:12:00.771893+0000","last_fullsized":"2024-03-04T12:12:00.771893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771263+0000","last_change":"2024-03-04T12:11:54.898528+0000","last_active":"2024-03-04T12:12:00.771263+0000","last_peered":"2024-03-04T12:12:00.771263+0000","last_clean":"2024-03-04T12:12:00.771263+0000","last_became_active":"2024-03-04T12:11:54.897521+0000","last_became_peered":"2024-03-04T12:11:54.897521+0000","last_unstale":"2024-03-04T12:12:00.771263+0000","last_undegraded":"2024-03-04T12:12:00.771263+0000","last_fullsized":"2024-03-04T12:12:00.771263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771266+0000","last_change":"2024-03-04T12:11:54.901718+0000","last_active":"2024-03-04T12:12:00.771266+0000","last_peered":"2024-03-04T12:12:00.771266+0000","last_clean":"2024-03-04T12:12:00.771266+0000","last_became_active":"2024-03-04T12:11:54.901395+0000","last_became_peered":"2024-03-04T12:11:54.901395+0000","last_unstale":"2024-03-04T12:12:00.771266+0000","last_undegraded":"2024-03-04T12:12:00.771266+0000","last_fullsized":"2024-03-04T12:12:00.771266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772905+0000","last_change":"2024-03-04T12:11:54.907257+0000","last_active":"2024-03-04T12:12:00.772905+0000","last_peered":"2024-03-04T12:12:00.772905+0000","last_clean":"2024-03-04T12:12:00.772905+0000","last_became_active":"2024-03-04T12:11:54.906648+0000","last_became_peered":"2024-03-04T12:11:54.906648+0000","last_unstale":"2024-03-04T12:12:00.772905+0000","last_undegraded":"2024-03-04T12:12:00.772905+0000","last_fullsized":"2024-03-04T12:12:00.772905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:02.739456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:12:17.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:12:18.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.060+0000 7f9ea83d4640 1 Processor -- start 2024-03-04T12:12:18.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.060+0000 7f9ea83d4640 1 -- start start 2024-03-04T12:12:18.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9ea0154a80 0x7f9ea015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ea00cf650 0x7f9ea00cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea01616c0 0x7f9ea0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00cdaa0 con 0x7f9ea00cf650 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00cdd80 con 0x7f9ea0154a80 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea83d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00ce060 con 0x7f9ea01616c0 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ea00cf650 0x7f9ea00cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ea00cf650 0x7f9ea00cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39752/0 (socket says 172.21.15.148:39752) 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 -- 172.21.15.148:0/3086456107 learned_addr learned my addr 172.21.15.148:0/3086456107 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea6971640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea01616c0 0x7f9ea0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 -- 172.21.15.148:0/3086456107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea01616c0 msgr2=0x7f9ea0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea01616c0 0x7f9ea0161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea6170640 1 --2- 172.21.15.148:0/3086456107 >> v2:172.21.15.191:3300/0 conn(0x7f9ea0154a80 0x7f9ea015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 -- 172.21.15.148:0/3086456107 >> v2:172.21.15.191:3300/0 conn(0x7f9ea0154a80 msgr2=0x7f9ea015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:18.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3086456107 >> v2:172.21.15.191:3300/0 conn(0x7f9ea0154a80 0x7f9ea015e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.061+0000 7f9ea596f640 1 -- 172.21.15.148:0/3086456107 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00ce340 con 0x7f9ea00cf650 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.062+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea00cfae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ea006e970 tx=0x7f9e9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f5729ae8bd124 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.062+0000 7f9ea516e640 1 -- 172.21.15.148:0/3086456107 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90035020 con 0x7f9ea00cf650 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.062+0000 7f9ea516e640 1 -- 172.21.15.148:0/3086456107 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90031870 con 0x7f9ea00cf650 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.062+0000 7f9ea516e640 1 -- 172.21.15.148:0/3086456107 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90031c40 con 0x7f9ea00cf650 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3086456107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 msgr2=0x7f9ea00cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea00cfae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ea006e970 tx=0x7f9e9002b910 comp rx=0 tx=0).stop 2024-03-04T12:12:18.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3086456107 shutdown_connections 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea01616c0 0x7f9ea0161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3086456107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea00cfae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3086456107 >> v2:172.21.15.191:3300/0 conn(0x7f9ea0154a80 0x7f9ea015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3086456107 >> 172.21.15.148:0/3086456107 conn(0x7f9ea006f5f0 msgr2=0x7f9ea006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3086456107 shutdown_connections 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.063+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3086456107 wait complete. 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 Processor -- start 2024-03-04T12:12:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 -- start start 2024-03-04T12:12:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea0154a80 0x7f9ea00b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 0x7f9ea00b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0154fe0 con 0x7f9ea00cf650 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0154e60 con 0x7f9ea01616c0 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.064+0000 7f9ea83d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00ceb40 con 0x7f9ea0154a80 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39756/0 (socket says 172.21.15.148:39756) 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 -- 172.21.15.148:0/3991966826 learned_addr learned my addr 172.21.15.148:0/3991966826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6971640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 0x7f9ea00b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea0154a80 0x7f9ea00b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 -- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea0154a80 msgr2=0x7f9ea00b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea0154a80 0x7f9ea00b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 -- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 msgr2=0x7f9ea00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 0x7f9ea00b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 -- 172.21.15.148:0/3991966826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea01f4730 con 0x7f9ea00cf650 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6971640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 0x7f9ea00b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea6170640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c820 tx=0x7f9e9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c015070 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c004220 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea00b5ae0 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ce10 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.065+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00c5920 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.066+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0074f60 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.067+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9e9c0077d0 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.067+0000 7f9e96ffd640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.067+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9e9c08a350 con 0x7f9ea00cf650 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.067+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.068+0000 7f9ea596f640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 0x7f9e8406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e90036000 tx=0x7f9e9003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.071+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e9c058e00 con 0x7f9ea00cf650 2024-03-04T12:12:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.239+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ea0071970 con 0x7f9e8406b4d0 2024-03-04T12:12:18.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.247+0000 7f9e96ffd640 1 -- 172.21.15.148:0/3991966826 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f9ea0071970 con 0x7f9e8406b4d0 2024-03-04T12:12:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:18.283 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 msgr2=0x7f9e8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 0x7f9e8406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e90036000 tx=0x7f9e9003c040 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 msgr2=0x7f9ea015faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c820 tx=0x7f9e9c00cce0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 shutdown_connections 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ea01616c0 0x7f9ea00b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ea0154a80 0x7f9ea00b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 --2- 172.21.15.148:0/3991966826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea00cf650 0x7f9ea015faf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 >> 172.21.15.148:0/3991966826 conn(0x7f9ea006f5f0 msgr2=0x7f9ea00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 shutdown_connections 2024-03-04T12:12:18.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.250+0000 7f9ea83d4640 1 -- 172.21.15.148:0/3991966826 wait complete. 2024-03-04T12:12:18.286 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-03-04T12:12:16.718598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771745+0000","last_change":"2024-03-04T12:11:53.892196+0000","last_active":"2024-03-04T12:12:00.771745+0000","last_peered":"2024-03-04T12:12:00.771745+0000","last_clean":"2024-03-04T12:12:00.771745+0000","last_became_active":"2024-03-04T12:11:53.891878+0000","last_became_peered":"2024-03-04T12:11:53.891878+0000","last_unstale":"2024-03-04T12:12:00.771745+0000","last_undegraded":"2024-03-04T12:12:00.771745+0000","last_fullsized":"2024-03-04T12:12:00.771745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:54.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921065+0000","last_change":"2024-03-04T12:11:54.905432+0000","last_active":"2024-03-04T12:11:59.921065+0000","last_peered":"2024-03-04T12:11:59.921065+0000","last_clean":"2024-03-04T12:11:59.921065+0000","last_became_active":"2024-03-04T12:11:54.905131+0000","last_became_peered":"2024-03-04T12:11:54.905131+0000","last_unstale":"2024-03-04T12:11:59.921065+0000","last_undegraded":"2024-03-04T12:11:59.921065+0000","last_fullsized":"2024-03-04T12:11:59.921065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920122+0000","last_change":"2024-03-04T12:11:58.938162+0000","last_active":"2024-03-04T12:11:59.920122+0000","last_peered":"2024-03-04T12:11:59.920122+0000","last_clean":"2024-03-04T12:11:59.920122+0000","last_became_active":"2024-03-04T12:11:58.937622+0000","last_became_peered":"2024-03-04T12:11:58.937622+0000","last_unstale":"2024-03-04T12:11:59.920122+0000","last_undegraded":"2024-03-04T12:11:59.920122+0000","last_fullsized":"2024-03-04T12:11:59.920122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:33:55.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921215+0000","last_change":"2024-03-04T12:11:53.881535+0000","last_active":"2024-03-04T12:11:59.921215+0000","last_peered":"2024-03-04T12:11:59.921215+0000","last_clean":"2024-03-04T12:11:59.921215+0000","last_became_active":"2024-03-04T12:11:53.880591+0000","last_became_peered":"2024-03-04T12:11:53.880591+0000","last_unstale":"2024-03-04T12:11:59.921215+0000","last_undegraded":"2024-03-04T12:11:59.921215+0000","last_fullsized":"2024-03-04T12:11:59.921215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771443+0000","last_change":"2024-03-04T12:11:54.906418+0000","last_active":"2024-03-04T12:12:00.771443+0000","last_peered":"2024-03-04T12:12:00.771443+0000","last_clean":"2024-03-04T12:12:00.771443+0000","last_became_active":"2024-03-04T12:11:54.906146+0000","last_became_peered":"2024-03-04T12:11:54.906146+0000","last_unstale":"2024-03-04T12:12:00.771443+0000","last_undegraded":"2024-03-04T12:12:00.771443+0000","last_fullsized":"2024-03-04T12:12:00.771443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771197+0000","last_change":"2024-03-04T12:11:58.945630+0000","last_active":"2024-03-04T12:12:00.771197+0000","last_peered":"2024-03-04T12:12:00.771197+0000","last_clean":"2024-03-04T12:12:00.771197+0000","last_became_active":"2024-03-04T12:11:58.945230+0000","last_became_peered":"2024-03-04T12:11:58.945230+0000","last_unstale":"2024-03-04T12:12:00.771197+0000","last_undegraded":"2024-03-04T12:12:00.771197+0000","last_fullsized":"2024-03-04T12:12:00.771197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:50.652152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.895431+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.895122+0000","last_became_peered":"2024-03-04T12:11:53.895122+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:54.330076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771997+0000","last_change":"2024-03-04T12:11:54.902410+0000","last_active":"2024-03-04T12:12:00.771997+0000","last_peered":"2024-03-04T12:12:00.771997+0000","last_clean":"2024-03-04T12:12:00.771997+0000","last_became_active":"2024-03-04T12:11:54.902131+0000","last_became_peered":"2024-03-04T12:11:54.902131+0000","last_unstale":"2024-03-04T12:12:00.771997+0000","last_undegraded":"2024-03-04T12:12:00.771997+0000","last_fullsized":"2024-03-04T12:12:00.771997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772908+0000","last_change":"2024-03-04T12:11:58.943323+0000","last_active":"2024-03-04T12:12:00.772908+0000","last_peered":"2024-03-04T12:12:00.772908+0000","last_clean":"2024-03-04T12:12:00.772908+0000","last_became_active":"2024-03-04T12:11:58.942878+0000","last_became_peered":"2024-03-04T12:11:58.942878+0000","last_unstale":"2024-03-04T12:12:00.772908+0000","last_undegraded":"2024-03-04T12:12:00.772908+0000","last_fullsized":"2024-03-04T12:12:00.772908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771622+0000","last_change":"2024-03-04T12:11:53.893583+0000","last_active":"2024-03-04T12:12:00.771622+0000","last_peered":"2024-03-04T12:12:00.771622+0000","last_clean":"2024-03-04T12:12:00.771622+0000","last_became_active":"2024-03-04T12:11:53.893295+0000","last_became_peered":"2024-03-04T12:11:53.893295+0000","last_unstale":"2024-03-04T12:12:00.771622+0000","last_undegraded":"2024-03-04T12:12:00.771622+0000","last_fullsized":"2024-03-04T12:12:00.771622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772748+0000","last_change":"2024-03-04T12:11:54.901339+0000","last_active":"2024-03-04T12:12:00.772748+0000","last_peered":"2024-03-04T12:12:00.772748+0000","last_clean":"2024-03-04T12:12:00.772748+0000","last_became_active":"2024-03-04T12:11:54.900914+0000","last_became_peered":"2024-03-04T12:11:54.900914+0000","last_unstale":"2024-03-04T12:12:00.772748+0000","last_undegraded":"2024-03-04T12:12:00.772748+0000","last_fullsized":"2024-03-04T12:12:00.772748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773092+0000","last_change":"2024-03-04T12:11:58.937633+0000","last_active":"2024-03-04T12:12:00.773092+0000","last_peered":"2024-03-04T12:12:00.773092+0000","last_clean":"2024-03-04T12:12:00.773092+0000","last_became_active":"2024-03-04T12:11:58.936727+0000","last_became_peered":"2024-03-04T12:11:58.936727+0000","last_unstale":"2024-03-04T12:12:00.773092+0000","last_undegraded":"2024-03-04T12:12:00.773092+0000","last_fullsized":"2024-03-04T12:12:00.773092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:03:00.255209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921104+0000","last_change":"2024-03-04T12:11:54.905529+0000","last_active":"2024-03-04T12:11:59.921104+0000","last_peered":"2024-03-04T12:11:59.921104+0000","last_clean":"2024-03-04T12:11:59.921104+0000","last_became_active":"2024-03-04T12:11:54.904590+0000","last_became_peered":"2024-03-04T12:11:54.904590+0000","last_unstale":"2024-03-04T12:11:59.921104+0000","last_undegraded":"2024-03-04T12:11:59.921104+0000","last_fullsized":"2024-03-04T12:11:59.921104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772191+0000","last_change":"2024-03-04T12:11:53.892967+0000","last_active":"2024-03-04T12:12:00.772191+0000","last_peered":"2024-03-04T12:12:00.772191+0000","last_clean":"2024-03-04T12:12:00.772191+0000","last_became_active":"2024-03-04T12:11:53.892698+0000","last_became_peered":"2024-03-04T12:11:53.892698+0000","last_unstale":"2024-03-04T12:12:00.772191+0000","last_undegraded":"2024-03-04T12:12:00.772191+0000","last_fullsized":"2024-03-04T12:12:00.772191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772796+0000","last_change":"2024-03-04T12:11:58.947133+0000","last_active":"2024-03-04T12:12:00.772796+0000","last_peered":"2024-03-04T12:12:00.772796+0000","last_clean":"2024-03-04T12:12:00.772796+0000","last_became_active":"2024-03-04T12:11:58.946617+0000","last_became_peered":"2024-03-04T12:11:58.946617+0000","last_unstale":"2024-03-04T12:12:00.772796+0000","last_undegraded":"2024-03-04T12:12:00.772796+0000","last_fullsized":"2024-03-04T12:12:00.772796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920487+0000","last_change":"2024-03-04T12:11:54.896623+0000","last_active":"2024-03-04T12:11:59.920487+0000","last_peered":"2024-03-04T12:11:59.920487+0000","last_clean":"2024-03-04T12:11:59.920487+0000","last_became_active":"2024-03-04T12:11:54.896359+0000","last_became_peered":"2024-03-04T12:11:54.896359+0000","last_unstale":"2024-03-04T12:11:59.920487+0000","last_undegraded":"2024-03-04T12:11:59.920487+0000","last_fullsized":"2024-03-04T12:11:59.920487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771403+0000","last_change":"2024-03-04T12:11:53.886825+0000","last_active":"2024-03-04T12:12:00.771403+0000","last_peered":"2024-03-04T12:12:00.771403+0000","last_clean":"2024-03-04T12:12:00.771403+0000","last_became_active":"2024-03-04T12:11:53.886525+0000","last_became_peered":"2024-03-04T12:11:53.886525+0000","last_unstale":"2024-03-04T12:12:00.771403+0000","last_undegraded":"2024-03-04T12:12:00.771403+0000","last_fullsized":"2024-03-04T12:12:00.771403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772607+0000","last_change":"2024-03-04T12:11:58.946327+0000","last_active":"2024-03-04T12:12:00.772607+0000","last_peered":"2024-03-04T12:12:00.772607+0000","last_clean":"2024-03-04T12:12:00.772607+0000","last_became_active":"2024-03-04T12:11:58.945962+0000","last_became_peered":"2024-03-04T12:11:58.945962+0000","last_unstale":"2024-03-04T12:12:00.772607+0000","last_undegraded":"2024-03-04T12:12:00.772607+0000","last_fullsized":"2024-03-04T12:12:00.772607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920261+0000","last_change":"2024-03-04T12:11:54.903139+0000","last_active":"2024-03-04T12:11:59.920261+0000","last_peered":"2024-03-04T12:11:59.920261+0000","last_clean":"2024-03-04T12:11:59.920261+0000","last_became_active":"2024-03-04T12:11:54.902809+0000","last_became_peered":"2024-03-04T12:11:54.902809+0000","last_unstale":"2024-03-04T12:11:59.920261+0000","last_undegraded":"2024-03-04T12:11:59.920261+0000","last_fullsized":"2024-03-04T12:11:59.920261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770887+0000","last_change":"2024-03-04T12:11:53.889802+0000","last_active":"2024-03-04T12:12:00.770887+0000","last_peered":"2024-03-04T12:12:00.770887+0000","last_clean":"2024-03-04T12:12:00.770887+0000","last_became_active":"2024-03-04T12:11:53.889550+0000","last_became_peered":"2024-03-04T12:11:53.889550+0000","last_unstale":"2024-03-04T12:12:00.770887+0000","last_undegraded":"2024-03-04T12:12:00.770887+0000","last_fullsized":"2024-03-04T12:12:00.770887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771267+0000","last_change":"2024-03-04T12:11:58.941310+0000","last_active":"2024-03-04T12:12:00.771267+0000","last_peered":"2024-03-04T12:12:00.771267+0000","last_clean":"2024-03-04T12:12:00.771267+0000","last_became_active":"2024-03-04T12:11:58.940932+0000","last_became_peered":"2024-03-04T12:11:58.940932+0000","last_unstale":"2024-03-04T12:12:00.771267+0000","last_undegraded":"2024-03-04T12:12:00.771267+0000","last_fullsized":"2024-03-04T12:12:00.771267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772213+0000","last_change":"2024-03-04T12:11:54.900654+0000","last_active":"2024-03-04T12:12:00.772213+0000","last_peered":"2024-03-04T12:12:00.772213+0000","last_clean":"2024-03-04T12:12:00.772213+0000","last_became_active":"2024-03-04T12:11:54.900311+0000","last_became_peered":"2024-03-04T12:11:54.900311+0000","last_unstale":"2024-03-04T12:12:00.772213+0000","last_undegraded":"2024-03-04T12:12:00.772213+0000","last_fullsized":"2024-03-04T12:12:00.772213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:23.687708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771071+0000","last_change":"2024-03-04T12:11:53.892220+0000","last_active":"2024-03-04T12:12:00.771071+0000","last_peered":"2024-03-04T12:12:00.771071+0000","last_clean":"2024-03-04T12:12:00.771071+0000","last_became_active":"2024-03-04T12:11:53.891859+0000","last_became_peered":"2024-03-04T12:11:53.891859+0000","last_unstale":"2024-03-04T12:12:00.771071+0000","last_undegraded":"2024-03-04T12:12:00.771071+0000","last_fullsized":"2024-03-04T12:12:00.771071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771968+0000","last_change":"2024-03-04T12:11:58.947661+0000","last_active":"2024-03-04T12:12:00.771968+0000","last_peered":"2024-03-04T12:12:00.771968+0000","last_clean":"2024-03-04T12:12:00.771968+0000","last_became_active":"2024-03-04T12:11:58.946889+0000","last_became_peered":"2024-03-04T12:11:58.946889+0000","last_unstale":"2024-03-04T12:12:00.771968+0000","last_undegraded":"2024-03-04T12:12:00.771968+0000","last_fullsized":"2024-03-04T12:12:00.771968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:48.995386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771351+0000","last_change":"2024-03-04T12:11:54.898183+0000","last_active":"2024-03-04T12:12:00.771351+0000","last_peered":"2024-03-04T12:12:00.771351+0000","last_clean":"2024-03-04T12:12:00.771351+0000","last_became_active":"2024-03-04T12:11:54.897873+0000","last_became_peered":"2024-03-04T12:11:54.897873+0000","last_unstale":"2024-03-04T12:12:00.771351+0000","last_undegraded":"2024-03-04T12:12:00.771351+0000","last_fullsized":"2024-03-04T12:12:00.771351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921618+0000","last_change":"2024-03-04T12:11:53.893093+0000","last_active":"2024-03-04T12:11:59.921618+0000","last_peered":"2024-03-04T12:11:59.921618+0000","last_clean":"2024-03-04T12:11:59.921618+0000","last_became_active":"2024-03-04T12:11:53.892821+0000","last_became_peered":"2024-03-04T12:11:53.892821+0000","last_unstale":"2024-03-04T12:11:59.921618+0000","last_undegraded":"2024-03-04T12:11:59.921618+0000","last_fullsized":"2024-03-04T12:11:59.921618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771969+0000","last_change":"2024-03-04T12:11:58.936275+0000","last_active":"2024-03-04T12:12:00.771969+0000","last_peered":"2024-03-04T12:12:00.771969+0000","last_clean":"2024-03-04T12:12:00.771969+0000","last_became_active":"2024-03-04T12:11:58.935879+0000","last_became_peered":"2024-03-04T12:11:58.935879+0000","last_unstale":"2024-03-04T12:12:00.771969+0000","last_undegraded":"2024-03-04T12:12:00.771969+0000","last_fullsized":"2024-03-04T12:12:00.771969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:57.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921287+0000","last_change":"2024-03-04T12:11:54.905996+0000","last_active":"2024-03-04T12:11:59.921287+0000","last_peered":"2024-03-04T12:11:59.921287+0000","last_clean":"2024-03-04T12:11:59.921287+0000","last_became_active":"2024-03-04T12:11:54.905164+0000","last_became_peered":"2024-03-04T12:11:54.905164+0000","last_unstale":"2024-03-04T12:11:59.921287+0000","last_undegraded":"2024-03-04T12:11:59.921287+0000","last_fullsized":"2024-03-04T12:11:59.921287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771364+0000","last_change":"2024-03-04T12:11:53.889023+0000","last_active":"2024-03-04T12:12:00.771364+0000","last_peered":"2024-03-04T12:12:00.771364+0000","last_clean":"2024-03-04T12:12:00.771364+0000","last_became_active":"2024-03-04T12:11:53.888732+0000","last_became_peered":"2024-03-04T12:11:53.888732+0000","last_unstale":"2024-03-04T12:12:00.771364+0000","last_undegraded":"2024-03-04T12:12:00.771364+0000","last_fullsized":"2024-03-04T12:12:00.771364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920041+0000","last_change":"2024-03-04T12:11:58.933844+0000","last_active":"2024-03-04T12:11:59.920041+0000","last_peered":"2024-03-04T12:11:59.920041+0000","last_clean":"2024-03-04T12:11:59.920041+0000","last_became_active":"2024-03-04T12:11:58.933456+0000","last_became_peered":"2024-03-04T12:11:58.933456+0000","last_unstale":"2024-03-04T12:11:59.920041+0000","last_undegraded":"2024-03-04T12:11:59.920041+0000","last_fullsized":"2024-03-04T12:11:59.920041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771789+0000","last_change":"2024-03-04T12:11:54.900251+0000","last_active":"2024-03-04T12:12:00.771789+0000","last_peered":"2024-03-04T12:12:00.771789+0000","last_clean":"2024-03-04T12:12:00.771789+0000","last_became_active":"2024-03-04T12:11:54.899988+0000","last_became_peered":"2024-03-04T12:11:54.899988+0000","last_unstale":"2024-03-04T12:12:00.771789+0000","last_undegraded":"2024-03-04T12:12:00.771789+0000","last_fullsized":"2024-03-04T12:12:00.771789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771966+0000","last_change":"2024-03-04T12:11:53.893557+0000","last_active":"2024-03-04T12:12:00.771966+0000","last_peered":"2024-03-04T12:12:00.771966+0000","last_clean":"2024-03-04T12:12:00.771966+0000","last_became_active":"2024-03-04T12:11:53.893170+0000","last_became_peered":"2024-03-04T12:11:53.893170+0000","last_unstale":"2024-03-04T12:12:00.771966+0000","last_undegraded":"2024-03-04T12:12:00.771966+0000","last_fullsized":"2024-03-04T12:12:00.771966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:04.682290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770797+0000","last_change":"2024-03-04T12:11:58.944095+0000","last_active":"2024-03-04T12:12:00.770797+0000","last_peered":"2024-03-04T12:12:00.770797+0000","last_clean":"2024-03-04T12:12:00.770797+0000","last_became_active":"2024-03-04T12:11:58.943691+0000","last_became_peered":"2024-03-04T12:11:58.943691+0000","last_unstale":"2024-03-04T12:12:00.770797+0000","last_undegraded":"2024-03-04T12:12:00.770797+0000","last_fullsized":"2024-03-04T12:12:00.770797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772840+0000","last_change":"2024-03-04T12:11:54.909256+0000","last_active":"2024-03-04T12:12:00.772840+0000","last_peered":"2024-03-04T12:12:00.772840+0000","last_clean":"2024-03-04T12:12:00.772840+0000","last_became_active":"2024-03-04T12:11:54.908971+0000","last_became_peered":"2024-03-04T12:11:54.908971+0000","last_unstale":"2024-03-04T12:12:00.772840+0000","last_undegraded":"2024-03-04T12:12:00.772840+0000","last_fullsized":"2024-03-04T12:12:00.772840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771832+0000","last_change":"2024-03-04T12:11:53.881754+0000","last_active":"2024-03-04T12:12:00.771832+0000","last_peered":"2024-03-04T12:12:00.771832+0000","last_clean":"2024-03-04T12:12:00.771832+0000","last_became_active":"2024-03-04T12:11:53.881149+0000","last_became_peered":"2024-03-04T12:11:53.881149+0000","last_unstale":"2024-03-04T12:12:00.771832+0000","last_undegraded":"2024-03-04T12:12:00.771832+0000","last_fullsized":"2024-03-04T12:12:00.771832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771018+0000","last_change":"2024-03-04T12:11:58.939152+0000","last_active":"2024-03-04T12:12:00.771018+0000","last_peered":"2024-03-04T12:12:00.771018+0000","last_clean":"2024-03-04T12:12:00.771018+0000","last_became_active":"2024-03-04T12:11:58.938820+0000","last_became_peered":"2024-03-04T12:11:58.938820+0000","last_unstale":"2024-03-04T12:12:00.771018+0000","last_undegraded":"2024-03-04T12:12:00.771018+0000","last_fullsized":"2024-03-04T12:12:00.771018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773199+0000","last_change":"2024-03-04T12:11:54.893605+0000","last_active":"2024-03-04T12:12:00.773199+0000","last_peered":"2024-03-04T12:12:00.773199+0000","last_clean":"2024-03-04T12:12:00.773199+0000","last_became_active":"2024-03-04T12:11:54.893164+0000","last_became_peered":"2024-03-04T12:11:54.893164+0000","last_unstale":"2024-03-04T12:12:00.773199+0000","last_undegraded":"2024-03-04T12:12:00.773199+0000","last_fullsized":"2024-03-04T12:12:00.773199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:32.197141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.884670+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.884396+0000","last_became_peered":"2024-03-04T12:11:53.884396+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771791+0000","last_change":"2024-03-04T12:11:58.939922+0000","last_active":"2024-03-04T12:12:00.771791+0000","last_peered":"2024-03-04T12:12:00.771791+0000","last_clean":"2024-03-04T12:12:00.771791+0000","last_became_active":"2024-03-04T12:11:58.939562+0000","last_became_peered":"2024-03-04T12:11:58.939562+0000","last_unstale":"2024-03-04T12:12:00.771791+0000","last_undegraded":"2024-03-04T12:12:00.771791+0000","last_fullsized":"2024-03-04T12:12:00.771791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770827+0000","last_change":"2024-03-04T12:11:54.898108+0000","last_active":"2024-03-04T12:12:00.770827+0000","last_peered":"2024-03-04T12:12:00.770827+0000","last_clean":"2024-03-04T12:12:00.770827+0000","last_became_active":"2024-03-04T12:11:54.897832+0000","last_became_peered":"2024-03-04T12:11:54.897832+0000","last_unstale":"2024-03-04T12:12:00.770827+0000","last_undegraded":"2024-03-04T12:12:00.770827+0000","last_fullsized":"2024-03-04T12:12:00.770827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921186+0000","last_change":"2024-03-04T12:11:53.884846+0000","last_active":"2024-03-04T12:11:59.921186+0000","last_peered":"2024-03-04T12:11:59.921186+0000","last_clean":"2024-03-04T12:11:59.921186+0000","last_became_active":"2024-03-04T12:11:53.884567+0000","last_became_peered":"2024-03-04T12:11:53.884567+0000","last_unstale":"2024-03-04T12:11:59.921186+0000","last_undegraded":"2024-03-04T12:11:59.921186+0000","last_fullsized":"2024-03-04T12:11:59.921186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771888+0000","last_change":"2024-03-04T12:11:58.936068+0000","last_active":"2024-03-04T12:12:00.771888+0000","last_peered":"2024-03-04T12:12:00.771888+0000","last_clean":"2024-03-04T12:12:00.771888+0000","last_became_active":"2024-03-04T12:11:58.935721+0000","last_became_peered":"2024-03-04T12:11:58.935721+0000","last_unstale":"2024-03-04T12:12:00.771888+0000","last_undegraded":"2024-03-04T12:12:00.771888+0000","last_fullsized":"2024-03-04T12:12:00.771888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770381+0000","last_change":"2024-03-04T12:11:54.893726+0000","last_active":"2024-03-04T12:12:00.770381+0000","last_peered":"2024-03-04T12:12:00.770381+0000","last_clean":"2024-03-04T12:12:00.770381+0000","last_became_active":"2024-03-04T12:11:54.893038+0000","last_became_peered":"2024-03-04T12:11:54.893038+0000","last_unstale":"2024-03-04T12:12:00.770381+0000","last_undegraded":"2024-03-04T12:12:00.770381+0000","last_fullsized":"2024-03-04T12:12:00.770381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770859+0000","last_change":"2024-03-04T12:11:53.893383+0000","last_active":"2024-03-04T12:12:00.770859+0000","last_peered":"2024-03-04T12:12:00.770859+0000","last_clean":"2024-03-04T12:12:00.770859+0000","last_became_active":"2024-03-04T12:11:53.892630+0000","last_became_peered":"2024-03-04T12:11:53.892630+0000","last_unstale":"2024-03-04T12:12:00.770859+0000","last_undegraded":"2024-03-04T12:12:00.770859+0000","last_fullsized":"2024-03-04T12:12:00.770859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771246+0000","last_change":"2024-03-04T12:11:58.946451+0000","last_active":"2024-03-04T12:12:00.771246+0000","last_peered":"2024-03-04T12:12:00.771246+0000","last_clean":"2024-03-04T12:12:00.771246+0000","last_became_active":"2024-03-04T12:11:58.946080+0000","last_became_peered":"2024-03-04T12:11:58.946080+0000","last_unstale":"2024-03-04T12:12:00.771246+0000","last_undegraded":"2024-03-04T12:12:00.771246+0000","last_fullsized":"2024-03-04T12:12:00.771246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920426+0000","last_change":"2024-03-04T12:11:54.902080+0000","last_active":"2024-03-04T12:11:59.920426+0000","last_peered":"2024-03-04T12:11:59.920426+0000","last_clean":"2024-03-04T12:11:59.920426+0000","last_became_active":"2024-03-04T12:11:54.901382+0000","last_became_peered":"2024-03-04T12:11:54.901382+0000","last_unstale":"2024-03-04T12:11:59.920426+0000","last_undegraded":"2024-03-04T12:11:59.920426+0000","last_fullsized":"2024-03-04T12:11:59.920426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772165+0000","last_change":"2024-03-04T12:11:53.883991+0000","last_active":"2024-03-04T12:12:00.772165+0000","last_peered":"2024-03-04T12:12:00.772165+0000","last_clean":"2024-03-04T12:12:00.772165+0000","last_became_active":"2024-03-04T12:11:53.881700+0000","last_became_peered":"2024-03-04T12:11:53.881700+0000","last_unstale":"2024-03-04T12:12:00.772165+0000","last_undegraded":"2024-03-04T12:12:00.772165+0000","last_fullsized":"2024-03-04T12:12:00.772165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770976+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:12:00.770976+0000","last_peered":"2024-03-04T12:12:00.770976+0000","last_clean":"2024-03-04T12:12:00.770976+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:12:00.770976+0000","last_undegraded":"2024-03-04T12:12:00.770976+0000","last_fullsized":"2024-03-04T12:12:00.770976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921000+0000","last_change":"2024-03-04T12:11:58.939590+0000","last_active":"2024-03-04T12:11:59.921000+0000","last_peered":"2024-03-04T12:11:59.921000+0000","last_clean":"2024-03-04T12:11:59.921000+0000","last_became_active":"2024-03-04T12:11:58.939177+0000","last_became_peered":"2024-03-04T12:11:58.939177+0000","last_unstale":"2024-03-04T12:11:59.921000+0000","last_undegraded":"2024-03-04T12:11:59.921000+0000","last_fullsized":"2024-03-04T12:11:59.921000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:54.906271+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:54.905428+0000","last_became_peered":"2024-03-04T12:11:54.905428+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921284+0000","last_change":"2024-03-04T12:11:53.885617+0000","last_active":"2024-03-04T12:11:59.921284+0000","last_peered":"2024-03-04T12:11:59.921284+0000","last_clean":"2024-03-04T12:11:59.921284+0000","last_became_active":"2024-03-04T12:11:53.885329+0000","last_became_peered":"2024-03-04T12:11:53.885329+0000","last_unstale":"2024-03-04T12:11:59.921284+0000","last_undegraded":"2024-03-04T12:11:59.921284+0000","last_fullsized":"2024-03-04T12:11:59.921284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921212+0000","last_change":"2024-03-04T12:11:58.946535+0000","last_active":"2024-03-04T12:11:59.921212+0000","last_peered":"2024-03-04T12:11:59.921212+0000","last_clean":"2024-03-04T12:11:59.921212+0000","last_became_active":"2024-03-04T12:11:58.946055+0000","last_became_peered":"2024-03-04T12:11:58.946055+0000","last_unstale":"2024-03-04T12:11:59.921212+0000","last_undegraded":"2024-03-04T12:11:59.921212+0000","last_fullsized":"2024-03-04T12:11:59.921212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772109+0000","last_change":"2024-03-04T12:11:54.910474+0000","last_active":"2024-03-04T12:12:00.772109+0000","last_peered":"2024-03-04T12:12:00.772109+0000","last_clean":"2024-03-04T12:12:00.772109+0000","last_became_active":"2024-03-04T12:11:54.909806+0000","last_became_peered":"2024-03-04T12:11:54.909806+0000","last_unstale":"2024-03-04T12:12:00.772109+0000","last_undegraded":"2024-03-04T12:12:00.772109+0000","last_fullsized":"2024-03-04T12:12:00.772109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771668+0000","last_change":"2024-03-04T12:11:53.882414+0000","last_active":"2024-03-04T12:12:00.771668+0000","last_peered":"2024-03-04T12:12:00.771668+0000","last_clean":"2024-03-04T12:12:00.771668+0000","last_became_active":"2024-03-04T12:11:53.882147+0000","last_became_peered":"2024-03-04T12:11:53.882147+0000","last_unstale":"2024-03-04T12:12:00.771668+0000","last_undegraded":"2024-03-04T12:12:00.771668+0000","last_fullsized":"2024-03-04T12:12:00.771668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771019+0000","last_change":"2024-03-04T12:11:58.946384+0000","last_active":"2024-03-04T12:12:00.771019+0000","last_peered":"2024-03-04T12:12:00.771019+0000","last_clean":"2024-03-04T12:12:00.771019+0000","last_became_active":"2024-03-04T12:11:58.945989+0000","last_became_peered":"2024-03-04T12:11:58.945989+0000","last_unstale":"2024-03-04T12:12:00.771019+0000","last_undegraded":"2024-03-04T12:12:00.771019+0000","last_fullsized":"2024-03-04T12:12:00.771019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772496+0000","last_change":"2024-03-04T12:11:54.903131+0000","last_active":"2024-03-04T12:12:00.772496+0000","last_peered":"2024-03-04T12:12:00.772496+0000","last_clean":"2024-03-04T12:12:00.772496+0000","last_became_active":"2024-03-04T12:11:54.902729+0000","last_became_peered":"2024-03-04T12:11:54.902729+0000","last_unstale":"2024-03-04T12:12:00.772496+0000","last_undegraded":"2024-03-04T12:12:00.772496+0000","last_fullsized":"2024-03-04T12:12:00.772496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:43.148028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921116+0000","last_change":"2024-03-04T12:11:53.884623+0000","last_active":"2024-03-04T12:11:59.921116+0000","last_peered":"2024-03-04T12:11:59.921116+0000","last_clean":"2024-03-04T12:11:59.921116+0000","last_became_active":"2024-03-04T12:11:53.884222+0000","last_became_peered":"2024-03-04T12:11:53.884222+0000","last_unstale":"2024-03-04T12:11:59.921116+0000","last_undegraded":"2024-03-04T12:11:59.921116+0000","last_fullsized":"2024-03-04T12:11:59.921116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771150+0000","last_change":"2024-03-04T12:11:58.935385+0000","last_active":"2024-03-04T12:12:00.771150+0000","last_peered":"2024-03-04T12:12:00.771150+0000","last_clean":"2024-03-04T12:12:00.771150+0000","last_became_active":"2024-03-04T12:11:58.934991+0000","last_became_peered":"2024-03-04T12:11:58.934991+0000","last_unstale":"2024-03-04T12:12:00.771150+0000","last_undegraded":"2024-03-04T12:12:00.771150+0000","last_fullsized":"2024-03-04T12:12:00.771150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:09.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773010+0000","last_change":"2024-03-04T12:11:54.901559+0000","last_active":"2024-03-04T12:12:00.773010+0000","last_peered":"2024-03-04T12:12:00.773010+0000","last_clean":"2024-03-04T12:12:00.773010+0000","last_became_active":"2024-03-04T12:11:54.901215+0000","last_became_peered":"2024-03-04T12:11:54.901215+0000","last_unstale":"2024-03-04T12:12:00.773010+0000","last_undegraded":"2024-03-04T12:12:00.773010+0000","last_fullsized":"2024-03-04T12:12:00.773010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772226+0000","last_change":"2024-03-04T12:11:53.884589+0000","last_active":"2024-03-04T12:12:00.772226+0000","last_peered":"2024-03-04T12:12:00.772226+0000","last_clean":"2024-03-04T12:12:00.772226+0000","last_became_active":"2024-03-04T12:11:53.884279+0000","last_became_peered":"2024-03-04T12:11:53.884279+0000","last_unstale":"2024-03-04T12:12:00.772226+0000","last_undegraded":"2024-03-04T12:12:00.772226+0000","last_fullsized":"2024-03-04T12:12:00.772226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:54.931530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771590+0000","last_change":"2024-03-04T12:11:58.945665+0000","last_active":"2024-03-04T12:12:00.771590+0000","last_peered":"2024-03-04T12:12:00.771590+0000","last_clean":"2024-03-04T12:12:00.771590+0000","last_became_active":"2024-03-04T12:11:58.945297+0000","last_became_peered":"2024-03-04T12:11:58.945297+0000","last_unstale":"2024-03-04T12:12:00.771590+0000","last_undegraded":"2024-03-04T12:12:00.771590+0000","last_fullsized":"2024-03-04T12:12:00.771590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:11.565262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771183+0000","last_change":"2024-03-04T12:11:54.902251+0000","last_active":"2024-03-04T12:12:00.771183+0000","last_peered":"2024-03-04T12:12:00.771183+0000","last_clean":"2024-03-04T12:12:00.771183+0000","last_became_active":"2024-03-04T12:11:54.901942+0000","last_became_peered":"2024-03-04T12:11:54.901942+0000","last_unstale":"2024-03-04T12:12:00.771183+0000","last_undegraded":"2024-03-04T12:12:00.771183+0000","last_fullsized":"2024-03-04T12:12:00.771183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920811+0000","last_change":"2024-03-04T12:11:53.883599+0000","last_active":"2024-03-04T12:11:59.920811+0000","last_peered":"2024-03-04T12:11:59.920811+0000","last_clean":"2024-03-04T12:11:59.920811+0000","last_became_active":"2024-03-04T12:11:53.883317+0000","last_became_peered":"2024-03-04T12:11:53.883317+0000","last_unstale":"2024-03-04T12:11:59.920811+0000","last_undegraded":"2024-03-04T12:11:59.920811+0000","last_fullsized":"2024-03-04T12:11:59.920811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771562+0000","last_change":"2024-03-04T12:11:58.936436+0000","last_active":"2024-03-04T12:12:00.771562+0000","last_peered":"2024-03-04T12:12:00.771562+0000","last_clean":"2024-03-04T12:12:00.771562+0000","last_became_active":"2024-03-04T12:11:58.935593+0000","last_became_peered":"2024-03-04T12:11:58.935593+0000","last_unstale":"2024-03-04T12:12:00.771562+0000","last_undegraded":"2024-03-04T12:12:00.771562+0000","last_fullsized":"2024-03-04T12:12:00.771562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920930+0000","last_change":"2024-03-04T12:11:54.908033+0000","last_active":"2024-03-04T12:11:59.920930+0000","last_peered":"2024-03-04T12:11:59.920930+0000","last_clean":"2024-03-04T12:11:59.920930+0000","last_became_active":"2024-03-04T12:11:54.907784+0000","last_became_peered":"2024-03-04T12:11:54.907784+0000","last_unstale":"2024-03-04T12:11:59.920930+0000","last_undegraded":"2024-03-04T12:11:59.920930+0000","last_fullsized":"2024-03-04T12:11:59.920930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921086+0000","last_change":"2024-03-04T12:11:53.889166+0000","last_active":"2024-03-04T12:11:59.921086+0000","last_peered":"2024-03-04T12:11:59.921086+0000","last_clean":"2024-03-04T12:11:59.921086+0000","last_became_active":"2024-03-04T12:11:53.888906+0000","last_became_peered":"2024-03-04T12:11:53.888906+0000","last_unstale":"2024-03-04T12:11:59.921086+0000","last_undegraded":"2024-03-04T12:11:59.921086+0000","last_fullsized":"2024-03-04T12:11:59.921086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921204+0000","last_change":"2024-03-04T12:11:58.944078+0000","last_active":"2024-03-04T12:11:59.921204+0000","last_peered":"2024-03-04T12:11:59.921204+0000","last_clean":"2024-03-04T12:11:59.921204+0000","last_became_active":"2024-03-04T12:11:58.942753+0000","last_became_peered":"2024-03-04T12:11:58.942753+0000","last_unstale":"2024-03-04T12:11:59.921204+0000","last_undegraded":"2024-03-04T12:11:59.921204+0000","last_fullsized":"2024-03-04T12:11:59.921204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:35.014220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772005+0000","last_change":"2024-03-04T12:11:54.902835+0000","last_active":"2024-03-04T12:12:00.772005+0000","last_peered":"2024-03-04T12:12:00.772005+0000","last_clean":"2024-03-04T12:12:00.772005+0000","last_became_active":"2024-03-04T12:11:54.902550+0000","last_became_peered":"2024-03-04T12:11:54.902550+0000","last_unstale":"2024-03-04T12:12:00.772005+0000","last_undegraded":"2024-03-04T12:12:00.772005+0000","last_fullsized":"2024-03-04T12:12:00.772005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:53.885828+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:53.885526+0000","last_became_peered":"2024-03-04T12:11:53.885526+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920921+0000","last_change":"2024-03-04T12:11:58.943710+0000","last_active":"2024-03-04T12:11:59.920921+0000","last_peered":"2024-03-04T12:11:59.920921+0000","last_clean":"2024-03-04T12:11:59.920921+0000","last_became_active":"2024-03-04T12:11:58.943297+0000","last_became_peered":"2024-03-04T12:11:58.943297+0000","last_unstale":"2024-03-04T12:11:59.920921+0000","last_undegraded":"2024-03-04T12:11:59.920921+0000","last_fullsized":"2024-03-04T12:11:59.920921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:56.606927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770672+0000","last_change":"2024-03-04T12:11:54.900415+0000","last_active":"2024-03-04T12:12:00.770672+0000","last_peered":"2024-03-04T12:12:00.770672+0000","last_clean":"2024-03-04T12:12:00.770672+0000","last_became_active":"2024-03-04T12:11:54.900101+0000","last_became_peered":"2024-03-04T12:11:54.900101+0000","last_unstale":"2024-03-04T12:12:00.770672+0000","last_undegraded":"2024-03-04T12:12:00.770672+0000","last_fullsized":"2024-03-04T12:12:00.770672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771029+0000","last_change":"2024-03-04T12:11:53.888493+0000","last_active":"2024-03-04T12:12:00.771029+0000","last_peered":"2024-03-04T12:12:00.771029+0000","last_clean":"2024-03-04T12:12:00.771029+0000","last_became_active":"2024-03-04T12:11:53.888223+0000","last_became_peered":"2024-03-04T12:11:53.888223+0000","last_unstale":"2024-03-04T12:12:00.771029+0000","last_undegraded":"2024-03-04T12:12:00.771029+0000","last_fullsized":"2024-03-04T12:12:00.771029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:40.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772694+0000","last_change":"2024-03-04T12:11:58.934500+0000","last_active":"2024-03-04T12:12:00.772694+0000","last_peered":"2024-03-04T12:12:00.772694+0000","last_clean":"2024-03-04T12:12:00.772694+0000","last_became_active":"2024-03-04T12:11:58.933910+0000","last_became_peered":"2024-03-04T12:11:58.933910+0000","last_unstale":"2024-03-04T12:12:00.772694+0000","last_undegraded":"2024-03-04T12:12:00.772694+0000","last_fullsized":"2024-03-04T12:12:00.772694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771718+0000","last_change":"2024-03-04T12:11:54.906835+0000","last_active":"2024-03-04T12:12:00.771718+0000","last_peered":"2024-03-04T12:12:00.771718+0000","last_clean":"2024-03-04T12:12:00.771718+0000","last_became_active":"2024-03-04T12:11:54.904477+0000","last_became_peered":"2024-03-04T12:11:54.904477+0000","last_unstale":"2024-03-04T12:12:00.771718+0000","last_undegraded":"2024-03-04T12:12:00.771718+0000","last_fullsized":"2024-03-04T12:12:00.771718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:04.123314+0000","last_change":"2024-03-04T12:11:53.889752+0000","last_active":"2024-03-04T12:12:04.123314+0000","last_peered":"2024-03-04T12:12:04.123314+0000","last_clean":"2024-03-04T12:12:04.123314+0000","last_became_active":"2024-03-04T12:11:53.889469+0000","last_became_peered":"2024-03-04T12:11:53.889469+0000","last_unstale":"2024-03-04T12:12:04.123314+0000","last_undegraded":"2024-03-04T12:12:04.123314+0000","last_fullsized":"2024-03-04T12:12:04.123314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771293+0000","last_change":"2024-03-04T12:11:58.941223+0000","last_active":"2024-03-04T12:12:00.771293+0000","last_peered":"2024-03-04T12:12:00.771293+0000","last_clean":"2024-03-04T12:12:00.771293+0000","last_became_active":"2024-03-04T12:11:58.940845+0000","last_became_peered":"2024-03-04T12:11:58.940845+0000","last_unstale":"2024-03-04T12:12:00.771293+0000","last_undegraded":"2024-03-04T12:12:00.771293+0000","last_fullsized":"2024-03-04T12:12:00.771293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772686+0000","last_change":"2024-03-04T12:11:54.902174+0000","last_active":"2024-03-04T12:12:00.772686+0000","last_peered":"2024-03-04T12:12:00.772686+0000","last_clean":"2024-03-04T12:12:00.772686+0000","last_became_active":"2024-03-04T12:11:54.901884+0000","last_became_peered":"2024-03-04T12:11:54.901884+0000","last_unstale":"2024-03-04T12:12:00.772686+0000","last_undegraded":"2024-03-04T12:12:00.772686+0000","last_fullsized":"2024-03-04T12:12:00.772686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771560+0000","last_change":"2024-03-04T12:11:53.892494+0000","last_active":"2024-03-04T12:12:00.771560+0000","last_peered":"2024-03-04T12:12:00.771560+0000","last_clean":"2024-03-04T12:12:00.771560+0000","last_became_active":"2024-03-04T12:11:53.892170+0000","last_became_peered":"2024-03-04T12:11:53.892170+0000","last_unstale":"2024-03-04T12:12:00.771560+0000","last_undegraded":"2024-03-04T12:12:00.771560+0000","last_fullsized":"2024-03-04T12:12:00.771560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773086+0000","last_change":"2024-03-04T12:11:58.945966+0000","last_active":"2024-03-04T12:12:00.773086+0000","last_peered":"2024-03-04T12:12:00.773086+0000","last_clean":"2024-03-04T12:12:00.773086+0000","last_became_active":"2024-03-04T12:11:58.945545+0000","last_became_peered":"2024-03-04T12:11:58.945545+0000","last_unstale":"2024-03-04T12:12:00.773086+0000","last_undegraded":"2024-03-04T12:12:00.773086+0000","last_fullsized":"2024-03-04T12:12:00.773086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771604+0000","last_change":"2024-03-04T12:11:54.900788+0000","last_active":"2024-03-04T12:12:00.771604+0000","last_peered":"2024-03-04T12:12:00.771604+0000","last_clean":"2024-03-04T12:12:00.771604+0000","last_became_active":"2024-03-04T12:11:54.900523+0000","last_became_peered":"2024-03-04T12:11:54.900523+0000","last_unstale":"2024-03-04T12:12:00.771604+0000","last_undegraded":"2024-03-04T12:12:00.771604+0000","last_fullsized":"2024-03-04T12:12:00.771604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773050+0000","last_change":"2024-03-04T12:11:53.887458+0000","last_active":"2024-03-04T12:12:00.773050+0000","last_peered":"2024-03-04T12:12:00.773050+0000","last_clean":"2024-03-04T12:12:00.773050+0000","last_became_active":"2024-03-04T12:11:53.887205+0000","last_became_peered":"2024-03-04T12:11:53.887205+0000","last_unstale":"2024-03-04T12:12:00.773050+0000","last_undegraded":"2024-03-04T12:12:00.773050+0000","last_fullsized":"2024-03-04T12:12:00.773050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920213+0000","last_change":"2024-03-04T12:11:58.944437+0000","last_active":"2024-03-04T12:11:59.920213+0000","last_peered":"2024-03-04T12:11:59.920213+0000","last_clean":"2024-03-04T12:11:59.920213+0000","last_became_active":"2024-03-04T12:11:58.944027+0000","last_became_peered":"2024-03-04T12:11:58.944027+0000","last_unstale":"2024-03-04T12:11:59.920213+0000","last_undegraded":"2024-03-04T12:11:59.920213+0000","last_fullsized":"2024-03-04T12:11:59.920213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:09:47.849893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773158+0000","last_change":"2024-03-04T12:11:54.904517+0000","last_active":"2024-03-04T12:12:00.773158+0000","last_peered":"2024-03-04T12:12:00.773158+0000","last_clean":"2024-03-04T12:12:00.773158+0000","last_became_active":"2024-03-04T12:11:54.904245+0000","last_became_peered":"2024-03-04T12:11:54.904245+0000","last_unstale":"2024-03-04T12:12:00.773158+0000","last_undegraded":"2024-03-04T12:12:00.773158+0000","last_fullsized":"2024-03-04T12:12:00.773158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921498+0000","last_change":"2024-03-04T12:11:53.888433+0000","last_active":"2024-03-04T12:11:59.921498+0000","last_peered":"2024-03-04T12:11:59.921498+0000","last_clean":"2024-03-04T12:11:59.921498+0000","last_became_active":"2024-03-04T12:11:53.888164+0000","last_became_peered":"2024-03-04T12:11:53.888164+0000","last_unstale":"2024-03-04T12:11:59.921498+0000","last_undegraded":"2024-03-04T12:11:59.921498+0000","last_fullsized":"2024-03-04T12:11:59.921498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920282+0000","last_change":"2024-03-04T12:11:58.937226+0000","last_active":"2024-03-04T12:11:59.920282+0000","last_peered":"2024-03-04T12:11:59.920282+0000","last_clean":"2024-03-04T12:11:59.920282+0000","last_became_active":"2024-03-04T12:11:58.936831+0000","last_became_peered":"2024-03-04T12:11:58.936831+0000","last_unstale":"2024-03-04T12:11:59.920282+0000","last_undegraded":"2024-03-04T12:11:59.920282+0000","last_fullsized":"2024-03-04T12:11:59.920282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:02.110563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770587+0000","last_change":"2024-03-04T12:11:58.946973+0000","last_active":"2024-03-04T12:12:00.770587+0000","last_peered":"2024-03-04T12:12:00.770587+0000","last_clean":"2024-03-04T12:12:00.770587+0000","last_became_active":"2024-03-04T12:11:58.946581+0000","last_became_peered":"2024-03-04T12:11:58.946581+0000","last_unstale":"2024-03-04T12:12:00.770587+0000","last_undegraded":"2024-03-04T12:12:00.770587+0000","last_fullsized":"2024-03-04T12:12:00.770587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770784+0000","last_change":"2024-03-04T12:11:54.900431+0000","last_active":"2024-03-04T12:12:00.770784+0000","last_peered":"2024-03-04T12:12:00.770784+0000","last_clean":"2024-03-04T12:12:00.770784+0000","last_became_active":"2024-03-04T12:11:54.900166+0000","last_became_peered":"2024-03-04T12:11:54.900166+0000","last_unstale":"2024-03-04T12:12:00.770784+0000","last_undegraded":"2024-03-04T12:12:00.770784+0000","last_fullsized":"2024-03-04T12:12:00.770784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:53.879757+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:53.878758+0000","last_became_peered":"2024-03-04T12:11:53.878758+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921468+0000","last_change":"2024-03-04T12:11:58.945414+0000","last_active":"2024-03-04T12:11:59.921468+0000","last_peered":"2024-03-04T12:11:59.921468+0000","last_clean":"2024-03-04T12:11:59.921468+0000","last_became_active":"2024-03-04T12:11:58.945031+0000","last_became_peered":"2024-03-04T12:11:58.945031+0000","last_unstale":"2024-03-04T12:11:59.921468+0000","last_undegraded":"2024-03-04T12:11:59.921468+0000","last_fullsized":"2024-03-04T12:11:59.921468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920217+0000","last_change":"2024-03-04T12:11:54.896991+0000","last_active":"2024-03-04T12:11:59.920217+0000","last_peered":"2024-03-04T12:11:59.920217+0000","last_clean":"2024-03-04T12:11:59.920217+0000","last_became_active":"2024-03-04T12:11:54.896725+0000","last_became_peered":"2024-03-04T12:11:54.896725+0000","last_unstale":"2024-03-04T12:11:59.920217+0000","last_undegraded":"2024-03-04T12:11:59.920217+0000","last_fullsized":"2024-03-04T12:11:59.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920530+0000","last_change":"2024-03-04T12:11:53.884328+0000","last_active":"2024-03-04T12:11:59.920530+0000","last_peered":"2024-03-04T12:11:59.920530+0000","last_clean":"2024-03-04T12:11:59.920530+0000","last_became_active":"2024-03-04T12:11:53.882865+0000","last_became_peered":"2024-03-04T12:11:53.882865+0000","last_unstale":"2024-03-04T12:11:59.920530+0000","last_undegraded":"2024-03-04T12:11:59.920530+0000","last_fullsized":"2024-03-04T12:11:59.920530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921283+0000","last_change":"2024-03-04T12:11:53.881932+0000","last_active":"2024-03-04T12:11:59.921283+0000","last_peered":"2024-03-04T12:11:59.921283+0000","last_clean":"2024-03-04T12:11:59.921283+0000","last_became_active":"2024-03-04T12:11:53.880843+0000","last_became_peered":"2024-03-04T12:11:53.880843+0000","last_unstale":"2024-03-04T12:11:59.921283+0000","last_undegraded":"2024-03-04T12:11:59.921283+0000","last_fullsized":"2024-03-04T12:11:59.921283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770941+0000","last_change":"2024-03-04T12:11:54.892168+0000","last_active":"2024-03-04T12:12:00.770941+0000","last_peered":"2024-03-04T12:12:00.770941+0000","last_clean":"2024-03-04T12:12:00.770941+0000","last_became_active":"2024-03-04T12:11:54.891886+0000","last_became_peered":"2024-03-04T12:11:54.891886+0000","last_unstale":"2024-03-04T12:12:00.770941+0000","last_undegraded":"2024-03-04T12:12:00.770941+0000","last_fullsized":"2024-03-04T12:12:00.770941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920785+0000","last_change":"2024-03-04T12:11:58.946859+0000","last_active":"2024-03-04T12:11:59.920785+0000","last_peered":"2024-03-04T12:11:59.920785+0000","last_clean":"2024-03-04T12:11:59.920785+0000","last_became_active":"2024-03-04T12:11:58.946495+0000","last_became_peered":"2024-03-04T12:11:58.946495+0000","last_unstale":"2024-03-04T12:11:59.920785+0000","last_undegraded":"2024-03-04T12:11:59.920785+0000","last_fullsized":"2024-03-04T12:11:59.920785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:25.320885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920580+0000","last_change":"2024-03-04T12:11:53.889151+0000","last_active":"2024-03-04T12:11:59.920580+0000","last_peered":"2024-03-04T12:11:59.920580+0000","last_clean":"2024-03-04T12:11:59.920580+0000","last_became_active":"2024-03-04T12:11:53.888882+0000","last_became_peered":"2024-03-04T12:11:53.888882+0000","last_unstale":"2024-03-04T12:11:59.920580+0000","last_undegraded":"2024-03-04T12:11:59.920580+0000","last_fullsized":"2024-03-04T12:11:59.920580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772788+0000","last_change":"2024-03-04T12:11:54.901800+0000","last_active":"2024-03-04T12:12:00.772788+0000","last_peered":"2024-03-04T12:12:00.772788+0000","last_clean":"2024-03-04T12:12:00.772788+0000","last_became_active":"2024-03-04T12:11:54.901502+0000","last_became_peered":"2024-03-04T12:11:54.901502+0000","last_unstale":"2024-03-04T12:12:00.772788+0000","last_undegraded":"2024-03-04T12:12:00.772788+0000","last_fullsized":"2024-03-04T12:12:00.772788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:36.238592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921109+0000","last_change":"2024-03-04T12:11:58.943607+0000","last_active":"2024-03-04T12:11:59.921109+0000","last_peered":"2024-03-04T12:11:59.921109+0000","last_clean":"2024-03-04T12:11:59.921109+0000","last_became_active":"2024-03-04T12:11:58.942837+0000","last_became_peered":"2024-03-04T12:11:58.942837+0000","last_unstale":"2024-03-04T12:11:59.921109+0000","last_undegraded":"2024-03-04T12:11:59.921109+0000","last_fullsized":"2024-03-04T12:11:59.921109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:59.670150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920801+0000","last_change":"2024-03-04T12:11:54.908034+0000","last_active":"2024-03-04T12:11:59.920801+0000","last_peered":"2024-03-04T12:11:59.920801+0000","last_clean":"2024-03-04T12:11:59.920801+0000","last_became_active":"2024-03-04T12:11:54.907761+0000","last_became_peered":"2024-03-04T12:11:54.907761+0000","last_unstale":"2024-03-04T12:11:59.920801+0000","last_undegraded":"2024-03-04T12:11:59.920801+0000","last_fullsized":"2024-03-04T12:11:59.920801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921098+0000","last_change":"2024-03-04T12:11:54.904118+0000","last_active":"2024-03-04T12:11:59.921098+0000","last_peered":"2024-03-04T12:11:59.921098+0000","last_clean":"2024-03-04T12:11:59.921098+0000","last_became_active":"2024-03-04T12:11:54.903747+0000","last_became_peered":"2024-03-04T12:11:54.903747+0000","last_unstale":"2024-03-04T12:11:59.921098+0000","last_undegraded":"2024-03-04T12:11:59.921098+0000","last_fullsized":"2024-03-04T12:11:59.921098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771638+0000","last_change":"2024-03-04T12:11:54.900197+0000","last_active":"2024-03-04T12:12:00.771638+0000","last_peered":"2024-03-04T12:12:00.771638+0000","last_clean":"2024-03-04T12:12:00.771638+0000","last_became_active":"2024-03-04T12:11:54.899939+0000","last_became_peered":"2024-03-04T12:11:54.899939+0000","last_unstale":"2024-03-04T12:12:00.771638+0000","last_undegraded":"2024-03-04T12:12:00.771638+0000","last_fullsized":"2024-03-04T12:12:00.771638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770848+0000","last_change":"2024-03-04T12:11:54.896614+0000","last_active":"2024-03-04T12:12:00.770848+0000","last_peered":"2024-03-04T12:12:00.770848+0000","last_clean":"2024-03-04T12:12:00.770848+0000","last_became_active":"2024-03-04T12:11:54.896312+0000","last_became_peered":"2024-03-04T12:11:54.896312+0000","last_unstale":"2024-03-04T12:12:00.770848+0000","last_undegraded":"2024-03-04T12:12:00.770848+0000","last_fullsized":"2024-03-04T12:12:00.770848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920990+0000","last_change":"2024-03-04T12:11:54.898597+0000","last_active":"2024-03-04T12:11:59.920990+0000","last_peered":"2024-03-04T12:11:59.920990+0000","last_clean":"2024-03-04T12:11:59.920990+0000","last_became_active":"2024-03-04T12:11:54.898339+0000","last_became_peered":"2024-03-04T12:11:54.898339+0000","last_unstale":"2024-03-04T12:11:59.920990+0000","last_undegraded":"2024-03-04T12:11:59.920990+0000","last_fullsized":"2024-03-04T12:11:59.920990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771385+0000","last_change":"2024-03-04T12:11:54.902225+0000","last_active":"2024-03-04T12:12:00.771385+0000","last_peered":"2024-03-04T12:12:00.771385+0000","last_clean":"2024-03-04T12:12:00.771385+0000","last_became_active":"2024-03-04T12:11:54.901968+0000","last_became_peered":"2024-03-04T12:11:54.901968+0000","last_unstale":"2024-03-04T12:12:00.771385+0000","last_undegraded":"2024-03-04T12:12:00.771385+0000","last_fullsized":"2024-03-04T12:12:00.771385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921621+0000","last_change":"2024-03-04T12:11:54.911542+0000","last_active":"2024-03-04T12:11:59.921621+0000","last_peered":"2024-03-04T12:11:59.921621+0000","last_clean":"2024-03-04T12:11:59.921621+0000","last_became_active":"2024-03-04T12:11:54.911032+0000","last_became_peered":"2024-03-04T12:11:54.911032+0000","last_unstale":"2024-03-04T12:11:59.921621+0000","last_undegraded":"2024-03-04T12:11:59.921621+0000","last_fullsized":"2024-03-04T12:11:59.921621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921271+0000","last_change":"2024-03-04T12:11:54.897959+0000","last_active":"2024-03-04T12:11:59.921271+0000","last_peered":"2024-03-04T12:11:59.921271+0000","last_clean":"2024-03-04T12:11:59.921271+0000","last_became_active":"2024-03-04T12:11:54.897665+0000","last_became_peered":"2024-03-04T12:11:54.897665+0000","last_unstale":"2024-03-04T12:11:59.921271+0000","last_undegraded":"2024-03-04T12:11:59.921271+0000","last_fullsized":"2024-03-04T12:11:59.921271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:54.899561+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:54.899289+0000","last_became_peered":"2024-03-04T12:11:54.899289+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920409+0000","last_change":"2024-03-04T12:11:54.897860+0000","last_active":"2024-03-04T12:11:59.920409+0000","last_peered":"2024-03-04T12:11:59.920409+0000","last_clean":"2024-03-04T12:11:59.920409+0000","last_became_active":"2024-03-04T12:11:54.897585+0000","last_became_peered":"2024-03-04T12:11:54.897585+0000","last_unstale":"2024-03-04T12:11:59.920409+0000","last_undegraded":"2024-03-04T12:11:59.920409+0000","last_fullsized":"2024-03-04T12:11:59.920409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771542+0000","last_change":"2024-03-04T12:11:54.904095+0000","last_active":"2024-03-04T12:12:00.771542+0000","last_peered":"2024-03-04T12:12:00.771542+0000","last_clean":"2024-03-04T12:12:00.771542+0000","last_became_active":"2024-03-04T12:11:54.903527+0000","last_became_peered":"2024-03-04T12:11:54.903527+0000","last_unstale":"2024-03-04T12:12:00.771542+0000","last_undegraded":"2024-03-04T12:12:00.771542+0000","last_fullsized":"2024-03-04T12:12:00.771542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770740+0000","last_change":"2024-03-04T12:11:54.904758+0000","last_active":"2024-03-04T12:12:00.770740+0000","last_peered":"2024-03-04T12:12:00.770740+0000","last_clean":"2024-03-04T12:12:00.770740+0000","last_became_active":"2024-03-04T12:11:54.904480+0000","last_became_peered":"2024-03-04T12:11:54.904480+0000","last_unstale":"2024-03-04T12:12:00.770740+0000","last_undegraded":"2024-03-04T12:12:00.770740+0000","last_fullsized":"2024-03-04T12:12:00.770740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921331+0000","last_change":"2024-03-04T12:11:54.901144+0000","last_active":"2024-03-04T12:11:59.921331+0000","last_peered":"2024-03-04T12:11:59.921331+0000","last_clean":"2024-03-04T12:11:59.921331+0000","last_became_active":"2024-03-04T12:11:54.900739+0000","last_became_peered":"2024-03-04T12:11:54.900739+0000","last_unstale":"2024-03-04T12:11:59.921331+0000","last_undegraded":"2024-03-04T12:11:59.921331+0000","last_fullsized":"2024-03-04T12:11:59.921331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771914+0000","last_change":"2024-03-04T12:11:54.905995+0000","last_active":"2024-03-04T12:12:00.771914+0000","last_peered":"2024-03-04T12:12:00.771914+0000","last_clean":"2024-03-04T12:12:00.771914+0000","last_became_active":"2024-03-04T12:11:54.904878+0000","last_became_peered":"2024-03-04T12:11:54.904878+0000","last_unstale":"2024-03-04T12:12:00.771914+0000","last_undegraded":"2024-03-04T12:12:00.771914+0000","last_fullsized":"2024-03-04T12:12:00.771914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921040+0000","last_change":"2024-03-04T12:11:54.904652+0000","last_active":"2024-03-04T12:11:59.921040+0000","last_peered":"2024-03-04T12:11:59.921040+0000","last_clean":"2024-03-04T12:11:59.921040+0000","last_became_active":"2024-03-04T12:11:54.904348+0000","last_became_peered":"2024-03-04T12:11:54.904348+0000","last_unstale":"2024-03-04T12:11:59.921040+0000","last_undegraded":"2024-03-04T12:11:59.921040+0000","last_fullsized":"2024-03-04T12:11:59.921040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:05.687532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772261+0000","last_change":"2024-03-04T12:11:54.905800+0000","last_active":"2024-03-04T12:12:00.772261+0000","last_peered":"2024-03-04T12:12:00.772261+0000","last_clean":"2024-03-04T12:12:00.772261+0000","last_became_active":"2024-03-04T12:11:54.905550+0000","last_became_peered":"2024-03-04T12:11:54.905550+0000","last_unstale":"2024-03-04T12:12:00.772261+0000","last_undegraded":"2024-03-04T12:12:00.772261+0000","last_fullsized":"2024-03-04T12:12:00.772261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771115+0000","last_change":"2024-03-04T12:11:54.909971+0000","last_active":"2024-03-04T12:12:00.771115+0000","last_peered":"2024-03-04T12:12:00.771115+0000","last_clean":"2024-03-04T12:12:00.771115+0000","last_became_active":"2024-03-04T12:11:54.909694+0000","last_became_peered":"2024-03-04T12:11:54.909694+0000","last_unstale":"2024-03-04T12:12:00.771115+0000","last_undegraded":"2024-03-04T12:12:00.771115+0000","last_fullsized":"2024-03-04T12:12:00.771115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770786+0000","last_change":"2024-03-04T12:11:54.901482+0000","last_active":"2024-03-04T12:12:00.770786+0000","last_peered":"2024-03-04T12:12:00.770786+0000","last_clean":"2024-03-04T12:12:00.770786+0000","last_became_active":"2024-03-04T12:11:54.901185+0000","last_became_peered":"2024-03-04T12:11:54.901185+0000","last_unstale":"2024-03-04T12:12:00.770786+0000","last_undegraded":"2024-03-04T12:12:00.770786+0000","last_fullsized":"2024-03-04T12:12:00.770786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770638+0000","last_change":"2024-03-04T12:11:54.902348+0000","last_active":"2024-03-04T12:12:00.770638+0000","last_peered":"2024-03-04T12:12:00.770638+0000","last_clean":"2024-03-04T12:12:00.770638+0000","last_became_active":"2024-03-04T12:11:54.902087+0000","last_became_peered":"2024-03-04T12:11:54.902087+0000","last_unstale":"2024-03-04T12:12:00.770638+0000","last_undegraded":"2024-03-04T12:12:00.770638+0000","last_fullsized":"2024-03-04T12:12:00.770638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:04:00.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920436+0000","last_change":"2024-03-04T12:11:54.904336+0000","last_active":"2024-03-04T12:11:59.920436+0000","last_peered":"2024-03-04T12:11:59.920436+0000","last_clean":"2024-03-04T12:11:59.920436+0000","last_became_active":"2024-03-04T12:11:54.904049+0000","last_became_peered":"2024-03-04T12:11:54.904049+0000","last_unstale":"2024-03-04T12:11:59.920436+0000","last_undegraded":"2024-03-04T12:11:59.920436+0000","last_fullsized":"2024-03-04T12:11:59.920436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772596+0000","last_change":"2024-03-04T12:11:54.907818+0000","last_active":"2024-03-04T12:12:00.772596+0000","last_peered":"2024-03-04T12:12:00.772596+0000","last_clean":"2024-03-04T12:12:00.772596+0000","last_became_active":"2024-03-04T12:11:54.907443+0000","last_became_peered":"2024-03-04T12:11:54.907443+0000","last_unstale":"2024-03-04T12:12:00.772596+0000","last_undegraded":"2024-03-04T12:12:00.772596+0000","last_fullsized":"2024-03-04T12:12:00.772596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:26.417645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921105+0000","last_change":"2024-03-04T12:11:54.906714+0000","last_active":"2024-03-04T12:11:59.921105+0000","last_peered":"2024-03-04T12:11:59.921105+0000","last_clean":"2024-03-04T12:11:59.921105+0000","last_became_active":"2024-03-04T12:11:54.906367+0000","last_became_peered":"2024-03-04T12:11:54.906367+0000","last_unstale":"2024-03-04T12:11:59.921105+0000","last_undegraded":"2024-03-04T12:11:59.921105+0000","last_fullsized":"2024-03-04T12:11:59.921105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921624+0000","last_change":"2024-03-04T12:11:54.908667+0000","last_active":"2024-03-04T12:11:59.921624+0000","last_peered":"2024-03-04T12:11:59.921624+0000","last_clean":"2024-03-04T12:11:59.921624+0000","last_became_active":"2024-03-04T12:11:54.908377+0000","last_became_peered":"2024-03-04T12:11:54.908377+0000","last_unstale":"2024-03-04T12:11:59.921624+0000","last_undegraded":"2024-03-04T12:11:59.921624+0000","last_fullsized":"2024-03-04T12:11:59.921624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772398+0000","last_change":"2024-03-04T12:11:54.907669+0000","last_active":"2024-03-04T12:12:00.772398+0000","last_peered":"2024-03-04T12:12:00.772398+0000","last_clean":"2024-03-04T12:12:00.772398+0000","last_became_active":"2024-03-04T12:11:54.905673+0000","last_became_peered":"2024-03-04T12:11:54.905673+0000","last_unstale":"2024-03-04T12:12:00.772398+0000","last_undegraded":"2024-03-04T12:12:00.772398+0000","last_fullsized":"2024-03-04T12:12:00.772398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:43.980065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920103+0000","last_change":"2024-03-04T12:11:54.904733+0000","last_active":"2024-03-04T12:11:59.920103+0000","last_peered":"2024-03-04T12:11:59.920103+0000","last_clean":"2024-03-04T12:11:59.920103+0000","last_became_active":"2024-03-04T12:11:54.903680+0000","last_became_peered":"2024-03-04T12:11:54.903680+0000","last_unstale":"2024-03-04T12:11:59.920103+0000","last_undegraded":"2024-03-04T12:11:59.920103+0000","last_fullsized":"2024-03-04T12:11:59.920103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771078+0000","last_change":"2024-03-04T12:11:54.902367+0000","last_active":"2024-03-04T12:12:00.771078+0000","last_peered":"2024-03-04T12:12:00.771078+0000","last_clean":"2024-03-04T12:12:00.771078+0000","last_became_active":"2024-03-04T12:11:54.902075+0000","last_became_peered":"2024-03-04T12:11:54.902075+0000","last_unstale":"2024-03-04T12:12:00.771078+0000","last_undegraded":"2024-03-04T12:12:00.771078+0000","last_fullsized":"2024-03-04T12:12:00.771078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771398+0000","last_change":"2024-03-04T12:11:54.899399+0000","last_active":"2024-03-04T12:12:00.771398+0000","last_peered":"2024-03-04T12:12:00.771398+0000","last_clean":"2024-03-04T12:12:00.771398+0000","last_became_active":"2024-03-04T12:11:54.899084+0000","last_became_peered":"2024-03-04T12:11:54.899084+0000","last_unstale":"2024-03-04T12:12:00.771398+0000","last_undegraded":"2024-03-04T12:12:00.771398+0000","last_fullsized":"2024-03-04T12:12:00.771398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772929+0000","last_change":"2024-03-04T12:11:54.903214+0000","last_active":"2024-03-04T12:12:00.772929+0000","last_peered":"2024-03-04T12:12:00.772929+0000","last_clean":"2024-03-04T12:12:00.772929+0000","last_became_active":"2024-03-04T12:11:54.901705+0000","last_became_peered":"2024-03-04T12:11:54.901705+0000","last_unstale":"2024-03-04T12:12:00.772929+0000","last_undegraded":"2024-03-04T12:12:00.772929+0000","last_fullsized":"2024-03-04T12:12:00.772929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771893+0000","last_change":"2024-03-04T12:11:54.910691+0000","last_active":"2024-03-04T12:12:00.771893+0000","last_peered":"2024-03-04T12:12:00.771893+0000","last_clean":"2024-03-04T12:12:00.771893+0000","last_became_active":"2024-03-04T12:11:54.910198+0000","last_became_peered":"2024-03-04T12:11:54.910198+0000","last_unstale":"2024-03-04T12:12:00.771893+0000","last_undegraded":"2024-03-04T12:12:00.771893+0000","last_fullsized":"2024-03-04T12:12:00.771893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771263+0000","last_change":"2024-03-04T12:11:54.898528+0000","last_active":"2024-03-04T12:12:00.771263+0000","last_peered":"2024-03-04T12:12:00.771263+0000","last_clean":"2024-03-04T12:12:00.771263+0000","last_became_active":"2024-03-04T12:11:54.897521+0000","last_became_peered":"2024-03-04T12:11:54.897521+0000","last_unstale":"2024-03-04T12:12:00.771263+0000","last_undegraded":"2024-03-04T12:12:00.771263+0000","last_fullsized":"2024-03-04T12:12:00.771263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771266+0000","last_change":"2024-03-04T12:11:54.901718+0000","last_active":"2024-03-04T12:12:00.771266+0000","last_peered":"2024-03-04T12:12:00.771266+0000","last_clean":"2024-03-04T12:12:00.771266+0000","last_became_active":"2024-03-04T12:11:54.901395+0000","last_became_peered":"2024-03-04T12:11:54.901395+0000","last_unstale":"2024-03-04T12:12:00.771266+0000","last_undegraded":"2024-03-04T12:12:00.771266+0000","last_fullsized":"2024-03-04T12:12:00.771266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772905+0000","last_change":"2024-03-04T12:11:54.907257+0000","last_active":"2024-03-04T12:12:00.772905+0000","last_peered":"2024-03-04T12:12:00.772905+0000","last_clean":"2024-03-04T12:12:00.772905+0000","last_became_active":"2024-03-04T12:11:54.906648+0000","last_became_peered":"2024-03-04T12:11:54.906648+0000","last_unstale":"2024-03-04T12:12:00.772905+0000","last_undegraded":"2024-03-04T12:12:00.772905+0000","last_fullsized":"2024-03-04T12:12:00.772905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:02.739456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:12:18.290 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-04T12:12:18.290 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T12:12:18.290 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-04T12:12:18.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-04T12:12:18.291 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-04T12:12:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 Processor -- start 2024-03-04T12:12:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 -- start start 2024-03-04T12:12:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ec15de00 0x7f86ec0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f86ec0b9380 0x7f86ec0b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec15e1e0 con 0x7f86ec0b70d0 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec0d2820 con 0x7f86ec15de00 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86f27ec640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec0d1c10 con 0x7f86ec0b9380 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.394+0000 7f86ebfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ec15de00 0x7f86ec0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ec15de00 0x7f86ec0b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39770/0 (socket says 172.21.15.148:39770) 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 -- 172.21.15.148:0/1820259119 learned_addr learned my addr 172.21.15.148:0/1820259119 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86eb7fe640 1 --2- 172.21.15.148:0/1820259119 >> v2:172.21.15.191:3300/0 conn(0x7f86ec0b9380 0x7f86ec0b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 -- 172.21.15.148:0/1820259119 >> v2:172.21.15.191:3300/0 conn(0x7f86ec0b9380 msgr2=0x7f86ec0b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 --2- 172.21.15.148:0/1820259119 >> v2:172.21.15.191:3300/0 conn(0x7f86ec0b9380 0x7f86ec0b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 -- 172.21.15.148:0/1820259119 >> v1:172.21.15.191:6789/0 conn(0x7f86ec0b70d0 legacy=0x7f86ec0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:18.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 -- 172.21.15.148:0/1820259119 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0d1ef0 con 0x7f86ec15de00 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.395+0000 7f86ebfff640 1 --2- 172.21.15.148:0/1820259119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b2e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86ec161830 tx=0x7f86d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c40a56d9f79855 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86eaffd640 1 -- 172.21.15.148:0/1820259119 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4035020 con 0x7f86ec15de00 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86eaffd640 1 -- 172.21.15.148:0/1820259119 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d4031870 con 0x7f86ec15de00 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86eaffd640 1 -- 172.21.15.148:0/1820259119 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4031c40 con 0x7f86ec15de00 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 -- 172.21.15.148:0/1820259119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 msgr2=0x7f86ec0b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 --2- 172.21.15.148:0/1820259119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b2e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86ec161830 tx=0x7f86d402b910 comp rx=0 tx=0).stop 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 -- 172.21.15.148:0/1820259119 shutdown_connections 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 --2- 172.21.15.148:0/1820259119 >> v2:172.21.15.191:3300/0 conn(0x7f86ec0b9380 0x7f86ec0b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 --2- 172.21.15.148:0/1820259119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b2e60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 -- 172.21.15.148:0/1820259119 >> 172.21.15.148:0/1820259119 conn(0x7f86ec0d0470 msgr2=0x7f86ec0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 -- 172.21.15.148:0/1820259119 shutdown_connections 2024-03-04T12:12:18.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.396+0000 7f86f27ec640 1 -- 172.21.15.148:0/1820259119 wait complete. 2024-03-04T12:12:18.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.397+0000 7f86f27ec640 1 Processor -- start 2024-03-04T12:12:18.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 -- start start 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86ec0b70d0 0x7f86ec1f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 0x7f86ec0b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0b97e0 con 0x7f86ec15de00 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0d24b0 con 0x7f86ec0b70d0 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f27ec640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0b5b80 con 0x7f86ec0b9380 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f0d89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f0d89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39786/0 (socket says 172.21.15.148:39786) 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.398+0000 7f86f0d89640 1 -- 172.21.15.148:0/3246926169 learned_addr learned my addr 172.21.15.148:0/3246926169 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86eb7fe640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 0x7f86ec0b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86ebfff640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86ec0b70d0 0x7f86ec1f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 -- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 msgr2=0x7f86ec0b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 0x7f86ec0b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 -- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86ec0b70d0 msgr2=0x7f86ec1f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86ec0b70d0 0x7f86ec1f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 -- 172.21.15.148:0/3246926169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0cbd20 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86eb7fe640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 0x7f86ec0b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f0d89640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c780 tx=0x7f86e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4017070 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4003da0 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec0cc000 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40052b0 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.399+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec0cc5c0 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.400+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec074f60 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.400+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f86e401a070 con 0x7f86ec15de00 2024-03-04T12:12:18.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.401+0000 7f86e8ff9640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 0x7f86cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.401+0000 7f86ebfff640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 0x7f86cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.401+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f86e4089bc0 con 0x7f86ec15de00 2024-03-04T12:12:18.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.401+0000 7f86ebfff640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 0x7f86cc06d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86d4036000 tx=0x7f86d403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.405+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86e4058700 con 0x7f86ec15de00 2024-03-04T12:12:18.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.620+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f86ec0cc790 con 0x7f86ec15de00 2024-03-04T12:12:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.623+0000 7f86e8ff9640 1 -- 172.21.15.148:0/3246926169 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171790 (secure 0 0 0) 0x7f86e4058000 con 0x7f86ec15de00 2024-03-04T12:12:18.627 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 msgr2=0x7f86cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 0x7f86cc06d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86d4036000 tx=0x7f86d403c040 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 msgr2=0x7f86ec0b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c780 tx=0x7f86e400cc40 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 shutdown_connections 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f86cc06b4d0 0x7f86cc06d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ec15de00 0x7f86ec0b8c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86ec0b9380 0x7f86ec0b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 --2- 172.21.15.148:0/3246926169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86ec0b70d0 0x7f86ec1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 >> 172.21.15.148:0/3246926169 conn(0x7f86ec0d0470 msgr2=0x7f86ec0b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 shutdown_connections 2024-03-04T12:12:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.626+0000 7f86f27ec640 1 -- 172.21.15.148:0/3246926169 wait complete. 2024-03-04T12:12:18.639 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2037319536},{"type":"v1","addr":"172.21.15.148:6833","nonce":2037319536}]},"active_addr":"172.21.15.148:6833/2037319536","active_change":"2024-03-04T12:11:34.705772+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":2518380340}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3657788960}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1788053658}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3560730806}]}]} 2024-03-04T12:12:18.645 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-04T12:12:18.645 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-04T12:12:18.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 Processor -- start 2024-03-04T12:12:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 -- start start 2024-03-04T12:12:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3cfc0b5b80 0x7f3cfc0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc15e1e0 con 0x7f3cfc0b2e60 2024-03-04T12:12:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0d1c10 con 0x7f3cfc15de00 2024-03-04T12:12:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.744+0000 7f3d01e81640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b98c0 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cfbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3cfc0b5b80 0x7f3cfc0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cf2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3cfc0b2e60 0x7f3cfc0b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47326/0 (socket says 172.21.15.148:47326) 2024-03-04T12:12:18.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cf2ffd640 1 -- 172.21.15.148:0/3554715236 learned_addr learned my addr 172.21.15.148:0/3554715236 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:18.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cfbfff640 1 -- 172.21.15.148:0/3554715236 >> v1:172.21.15.191:6789/0 conn(0x7f3cfc15de00 legacy=0x7f3cfc0b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:18.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cfbfff640 1 -- 172.21.15.148:0/3554715236 >> v1:172.21.15.148:6789/0 conn(0x7f3cfc0b2e60 legacy=0x7f3cfc0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.745+0000 7f3cfbfff640 1 -- 172.21.15.148:0/3554715236 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0d1ef0 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3cfbfff640 1 --2- 172.21.15.148:0/3554715236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc0b9380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009dc0 tx=0x7f3cf402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77130c4baace345b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3cfaffd640 1 -- 172.21.15.148:0/3554715236 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4039070 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3cfaffd640 1 -- 172.21.15.148:0/3554715236 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf402cc40 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3cfaffd640 1 -- 172.21.15.148:0/3554715236 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4034440 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 -- 172.21.15.148:0/3554715236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc0b5b80 msgr2=0x7f3cfc0b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 --2- 172.21.15.148:0/3554715236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc0b9380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009dc0 tx=0x7f3cf402be90 comp rx=0 tx=0).stop 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 -- 172.21.15.148:0/3554715236 shutdown_connections 2024-03-04T12:12:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 --2- 172.21.15.148:0/3554715236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc0b9380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 -- 172.21.15.148:0/3554715236 >> 172.21.15.148:0/3554715236 conn(0x7f3cfc0d0470 msgr2=0x7f3cfc0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 -- 172.21.15.148:0/3554715236 shutdown_connections 2024-03-04T12:12:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.746+0000 7f3d01e81640 1 -- 172.21.15.148:0/3554715236 wait complete. 2024-03-04T12:12:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.747+0000 7f3d01e81640 1 Processor -- start 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 -- start start 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc0b2e60 0x7f3cfc1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0d2820 con 0x7f3cfc15de00 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0d24b0 con 0x7f3cfc0b5b80 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3d01e81640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b72a0 con 0x7f3cfc0b2e60 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39808/0 (socket says 172.21.15.148:39808) 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3cfbfff640 1 -- 172.21.15.148:0/305178287 learned_addr learned my addr 172.21.15.148:0/305178287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.748+0000 7f3cfb7fe640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc0b2e60 0x7f3cfc1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 -- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc0b2e60 msgr2=0x7f3cfc1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc0b2e60 0x7f3cfc1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 -- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc0b5b80 msgr2=0x7f3cfc160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 -- 172.21.15.148:0/305178287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0cc980 con 0x7f3cfc15de00 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cfbfff640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009ee0 tx=0x7f3cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4039050 con 0x7f3cfc15de00 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4045070 con 0x7f3cfc15de00 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc0ccc60 con 0x7f3cfc15de00 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf403e830 con 0x7f3cfc15de00 2024-03-04T12:12:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.749+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc0cc610 con 0x7f3cfc15de00 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.750+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc074f60 con 0x7f3cfc15de00 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.751+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3cf403e9d0 con 0x7f3cfc15de00 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.751+0000 7f3cf8ff9640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 0x7f3ce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.751+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3cf40ad2a0 con 0x7f3cfc15de00 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.751+0000 7f3cfb7fe640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 0x7f3ce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.751+0000 7f3cfb7fe640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 0x7f3ce006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0cd3a0 tx=0x7f3cec007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.755+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cf407b980 con 0x7f3cfc15de00 2024-03-04T12:12:18.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.931+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cfc0b80d0 con 0x7f3cfc15de00 2024-03-04T12:12:18.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.932+0000 7f3cf8ff9640 1 -- 172.21.15.148:0/305178287 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f3cf407fa20 con 0x7f3cfc15de00 2024-03-04T12:12:18.935 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:18.935 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 msgr2=0x7f3ce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 0x7f3ce006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0cd3a0 tx=0x7f3cec007e30 comp rx=0 tx=0).stop 2024-03-04T12:12:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 msgr2=0x7f3cfc0c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009ee0 tx=0x7f3cf4004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 shutdown_connections 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce006b4d0 0x7f3ce006d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cfc15de00 0x7f3cfc0c9580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc0b5b80 0x7f3cfc160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 --2- 172.21.15.148:0/305178287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc0b2e60 0x7f3cfc1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.934+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 >> 172.21.15.148:0/305178287 conn(0x7f3cfc0d0470 msgr2=0x7f3cfc0b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.935+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 shutdown_connections 2024-03-04T12:12:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:18.935+0000 7f3d01e81640 1 -- 172.21.15.148:0/305178287 wait complete. 2024-03-04T12:12:18.948 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-04T12:12:18.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:19.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.047+0000 7f8524129640 1 Processor -- start 2024-03-04T12:12:19.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.047+0000 7f8524129640 1 -- start start 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f8524129640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15de00 0x7f851c0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f8524129640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851c0b9380 0x7f851c0b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f8524129640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c15e1e0 con 0x7f851c0b70d0 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f8524129640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0d2820 con 0x7f851c0b9380 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f8524129640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0d1c10 con 0x7f851c15de00 2024-03-04T12:12:19.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f85216c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851c0b9380 0x7f851c0b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f85226c6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f851c0b70d0 0x7f851c0b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47342/0 (socket says 172.21.15.148:47342) 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.048+0000 7f85226c6640 1 -- 172.21.15.148:0/869129012 learned_addr learned my addr 172.21.15.148:0/869129012 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f8521ec5640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15de00 0x7f851c0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f85216c4640 1 -- 172.21.15.148:0/869129012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15de00 msgr2=0x7f851c0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f85216c4640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15de00 0x7f851c0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f85216c4640 1 -- 172.21.15.148:0/869129012 >> v1:172.21.15.148:6789/0 conn(0x7f851c0b70d0 legacy=0x7f851c0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f85216c4640 1 -- 172.21.15.148:0/869129012 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0d1ef0 con 0x7f851c0b9380 2024-03-04T12:12:19.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f8520ec3640 1 -- 172.21.15.148:0/869129012 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412414111 0 0) 0x7f851c15e1e0 con 0x7f851c0b70d0 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.049+0000 7f85216c4640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b9380 0x7f851c0b6b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097a0 tx=0x7f850c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f25783e8d731c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8520ec3640 1 -- 172.21.15.148:0/869129012 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c035020 con 0x7f851c0b9380 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8520ec3640 1 -- 172.21.15.148:0/869129012 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c031830 con 0x7f851c0b9380 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8520ec3640 1 -- 172.21.15.148:0/869129012 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c031c00 con 0x7f851c0b9380 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 -- 172.21.15.148:0/869129012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b9380 msgr2=0x7f851c0b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b9380 0x7f851c0b6b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097a0 tx=0x7f850c02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 -- 172.21.15.148:0/869129012 shutdown_connections 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b9380 0x7f851c0b6b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 --2- 172.21.15.148:0/869129012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15de00 0x7f851c0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 -- 172.21.15.148:0/869129012 >> 172.21.15.148:0/869129012 conn(0x7f851c0d0470 msgr2=0x7f851c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 -- 172.21.15.148:0/869129012 shutdown_connections 2024-03-04T12:12:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.050+0000 7f8524129640 1 -- 172.21.15.148:0/869129012 wait complete. 2024-03-04T12:12:19.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.051+0000 7f8524129640 1 Processor -- start 2024-03-04T12:12:19.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.051+0000 7f8524129640 1 -- start start 2024-03-04T12:12:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 0x7f851c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 0x7f851c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b97e0 con 0x7f851c0b70d0 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0d24b0 con 0x7f851c15de00 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8524129640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b5b80 con 0x7f851c0b9380 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39828/0 (socket says 172.21.15.148:39828) 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 -- 172.21.15.148:0/30433900 learned_addr learned my addr 172.21.15.148:0/30433900 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f85216c4640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 0x7f851c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f85226c6640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 0x7f851c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 -- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 msgr2=0x7f851c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 0x7f851c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 -- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 msgr2=0x7f851c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 0x7f851c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f8521ec5640 1 -- 172.21.15.148:0/30433900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c1f47a0 con 0x7f851c0b70d0 2024-03-04T12:12:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f85216c4640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 0x7f851c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.052+0000 7f85226c6640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 0x7f851c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f8521ec5640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f851800d5f0 tx=0x7f851800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180125f0 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518019070 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c1f44c0 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180129d0 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.053+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c1f7240 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.054+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c0b8db0 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.054+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8518012b70 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.055+0000 7f85127fc640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.055+0000 7f85216c4640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.055+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f851808a8c0 con 0x7f851c0b70d0 2024-03-04T12:12:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.055+0000 7f85216c4640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 0x7f850006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f850c036000 tx=0x7f850c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.059+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f851c0b8db0 con 0x7f851c0b70d0 2024-03-04T12:12:19.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.230+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f851c074f60 con 0x7f851c0b70d0 2024-03-04T12:12:19.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.231+0000 7f85127fc640 1 -- 172.21.15.148:0/30433900 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f8518059490 con 0x7f851c0b70d0 2024-03-04T12:12:19.234 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:19.234 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 msgr2=0x7f850006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 0x7f850006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f850c036000 tx=0x7f850c008030 comp rx=0 tx=0).stop 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 msgr2=0x7f851c14dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f851800d5f0 tx=0x7f851800dab0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 shutdown_connections 2024-03-04T12:12:19.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c15de00 0x7f851c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0b9380 0x7f851c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 --2- 172.21.15.148:0/30433900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0b70d0 0x7f851c14dd00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.233+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 >> 172.21.15.148:0/30433900 conn(0x7f851c0d0470 msgr2=0x7f851c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.234+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 shutdown_connections 2024-03-04T12:12:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.234+0000 7f8524129640 1 -- 172.21.15.148:0/30433900 wait complete. 2024-03-04T12:12:19.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-04T12:12:19.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-04T12:12:19.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-04T12:12:19.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-04T12:12:19.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-04T12:12:19.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-04T12:12:19.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-04T12:12:19.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-04T12:12:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.418+0000 7f7871aea640 1 Processor -- start 2024-03-04T12:12:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.418+0000 7f67ed4b1640 1 Processor -- start 2024-03-04T12:12:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.418+0000 7f7871aea640 1 -- start start 2024-03-04T12:12:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.418+0000 7f67ed4b1640 1 -- start start 2024-03-04T12:12:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f67ed4b1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 0x7f67e80cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f7871aea640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f7871aea640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 0x7f786c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f7871aea640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0ccad0 con 0x7f786c15b960 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f7871aea640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b5190 con 0x7f786c0b47c0 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f7871aea640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b10a0 con 0x7f786c0b0700 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f74effff640 1 Processor -- start 2024-03-04T12:12:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f74effff640 1 -- start start 2024-03-04T12:12:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f67ed4b1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f67e80c3750 0x7f67e80c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f67ed4b1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e806fe00 con 0x7f67e81621f0 2024-03-04T12:12:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f67ed4b1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e80cf2c0 con 0x7f67e80c3750 2024-03-04T12:12:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.419+0000 7f67ed4b1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806e650 con 0x7f67e80cee60 2024-03-04T12:12:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e6ffd640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f67e81621f0 0x7f67e806f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58584/0 (socket says 172.21.15.148:58584) 2024-03-04T12:12:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e6ffd640 1 -- 172.21.15.148:0/3394397769 learned_addr learned my addr 172.21.15.148:0/3394397769 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38462/0 (socket says 172.21.15.148:38462) 2024-03-04T12:12:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 -- 172.21.15.148:0/1226424022 learned_addr learned my addr 172.21.15.148:0/1226424022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3394397769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 0x7f67e80cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e5ffb640 1 -- 172.21.15.148:0/3394397769 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132304782 0 0) 0x7f67d8001110 con 0x7f67e81621f0 2024-03-04T12:12:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e5ffb640 1 -- 172.21.15.148:0/3394397769 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d0003520 con 0x7f67e81621f0 2024-03-04T12:12:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786bfff640 1 --2- 172.21.15.148:0/1226424022 >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 0x7f786c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 -- 172.21.15.148:0/1226424022 >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 msgr2=0x7f786c0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f74effff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f74e800c090 0x7f74e8038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f74effff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8053c60 0x7f74e80df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 --2- 172.21.15.148:0/1226424022 >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 0x7f786c0b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 -- 172.21.15.148:0/1226424022 >> v1:172.21.15.191:6789/0 conn(0x7f786c15b960 legacy=0x7f786c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f786affd640 1 -- 172.21.15.148:0/1226424022 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0cfab0 con 0x7f786c0b0700 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7f786bfff640 1 --2- 172.21.15.148:0/1226424022 >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 0x7f786c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f786affd640 1 --2- 172.21.15.148:0/1226424022 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f786402bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad035c26bb2c0f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f786a7fc640 1 -- 172.21.15.148:0/1226424022 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786402c830 con 0x7f786c0b0700 2024-03-04T12:12:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7fa70149a640 1 Processor -- start 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7ffb8fd7e640 1 Processor -- start 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f74effff640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e80df750 con 0x7f74e8033ee0 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f74effff640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80544b0 con 0x7f74e800c090 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f74effff640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80541d0 con 0x7f74e8053c60 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7f74ee7fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8053c60 0x7f74e80df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74eeffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f74e800c090 0x7f74e8038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74ef7fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f74e8033ee0 0x7f74e804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58600/0 (socket says 172.21.15.148:58600) 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74ef7fe640 1 -- 172.21.15.148:0/1571856443 learned_addr learned my addr 172.21.15.148:0/1571856443 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7f42f800d640 1 Processor -- start 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e67fc640 1 -- 172.21.15.148:0/3394397769 >> v2:172.21.15.191:3300/0 conn(0x7f67e80c3750 msgr2=0x7f67e80c3be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3394397769 >> v2:172.21.15.191:3300/0 conn(0x7f67e80c3750 0x7f67e80c3be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e67fc640 1 -- 172.21.15.148:0/3394397769 >> v1:172.21.15.191:6789/0 conn(0x7f67e81621f0 legacy=0x7f67e806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.420+0000 7f67e67fc640 1 -- 172.21.15.148:0/3394397769 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e806e930 con 0x7f67e80cee60 2024-03-04T12:12:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3394397769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 0x7f67e80cfe40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009d90 tx=0x7f67e002be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e8762aa1ab4266 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f67e5ffb640 1 -- 172.21.15.148:0/3394397769 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0039070 con 0x7f67e80cee60 2024-03-04T12:12:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7fc086001640 1 Processor -- start 2024-03-04T12:12:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7fc086001640 1 -- start start 2024-03-04T12:12:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f786a7fc640 1 -- 172.21.15.148:0/1226424022 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786402c9d0 con 0x7f786c0b0700 2024-03-04T12:12:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f67e5ffb640 1 -- 172.21.15.148:0/3394397769 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e002cbe0 con 0x7f67e80cee60 2024-03-04T12:12:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f786a7fc640 1 -- 172.21.15.148:0/1226424022 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786402cda0 con 0x7f786c0b0700 2024-03-04T12:12:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f67e5ffb640 1 -- 172.21.15.148:0/3394397769 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0034440 con 0x7f67e80cee60 2024-03-04T12:12:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7fa70149a640 1 -- start start 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fa70149a640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa6f4055300 0x7fa6f40068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fa70149a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f4055770 con 0x7fa6f40eb060 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fa70149a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40387e0 con 0x7fa6f40ebb50 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fa70149a640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f4057e30 con 0x7fa6f4055300 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.421+0000 7ffb8fd7e640 1 -- start start 2024-03-04T12:12:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7ffb8fd7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb880b2720 0x7ffb88071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74eeffd640 1 -- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8053c60 msgr2=0x7f74e80df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74eeffd640 1 --2- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8053c60 0x7f74e80df210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74eeffd640 1 -- 172.21.15.148:0/1571856443 >> v1:172.21.15.191:6789/0 conn(0x7f74e8033ee0 legacy=0x7f74e804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f42f800d640 1 -- start start 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fc086001640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc0800cfd60 0x7fc080162700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fc086001640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fc086001640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc080070f80 con 0x7fc08015c740 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7fc086001640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0800718b0 con 0x7fc0800cfd60 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fc086001640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc080071260 con 0x7fc080162c40 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8db1a640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ffb880aead0 0x7ffb88162590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58630/0 (socket says 172.21.15.148:58630) 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fb27e91b640 1 Processor -- start 2024-03-04T12:12:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fb27e91b640 1 -- start start 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27e91b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb270009e20 0x7fb270056120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27e91b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb270056660 con 0x7fb27000a200 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27e91b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700553a0 con 0x7fb27003a230 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27e91b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb270055c90 con 0x7fb270009e20 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 -- 172.21.15.148:0/1226424022 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 msgr2=0x7f786c0b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 --2- 172.21.15.148:0/1226424022 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f786402bd10 comp rx=0 tx=0).stop 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 -- 172.21.15.148:0/1226424022 shutdown_connections 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 --2- 172.21.15.148:0/1226424022 >> v2:172.21.15.191:3300/0 conn(0x7f786c0b47c0 0x7f786c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 --2- 172.21.15.148:0/1226424022 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c0b0700 0x7f786c0b0b60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f7871aea640 1 -- 172.21.15.148:0/1226424022 >> 172.21.15.148:0/1226424022 conn(0x7f786c06f5f0 msgr2=0x7f786c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fa6fb7fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fa6f40ebb50 0x7fa6f40061a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58608/0 (socket says 172.21.15.148:58608) 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/2923480286 learned_addr learned my addr 172.21.15.148:0/2923480286 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fa700c99640 1 --2- 172.21.15.148:0/2923480286 >> v2:172.21.15.191:3300/0 conn(0x7fa6f4055300 0x7fa6f40068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325069012 0 0) 0x7fa6f4055770 con 0x7fa6f40eb060 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ec003520 con 0x7fa6f40eb060 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421865953 0 0) 0x7fa6e8001110 con 0x7fa6f40ebb50 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f4055770 con 0x7fa6f40ebb50 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873598861 0 0) 0x7fa6ec003520 con 0x7fa6f40eb060 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 >> v2:172.21.15.191:3300/0 conn(0x7fa6f4055300 msgr2=0x7fa6f40068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 --2- 172.21.15.148:0/2923480286 >> v2:172.21.15.191:3300/0 conn(0x7fa6f4055300 0x7fa6f40068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 >> v1:172.21.15.191:6789/0 conn(0x7fa6f40ebb50 legacy=0x7fa6f40061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4056d90 con 0x7fa6f40eb060 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fa6fc04f8e0 con 0x7fa6f40eb060 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fa700c99640 1 --2- 172.21.15.148:0/2923480286 >> v2:172.21.15.191:3300/0 conn(0x7fa6f4055300 0x7fa6f40068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8db1a640 1 -- 172.21.15.148:0/951921167 learned_addr learned my addr 172.21.15.148:0/951921167 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb88162de0 con 0x7ffb88071fc0 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb881630c0 con 0x7ffb880aead0 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb881633a0 con 0x7ffb880b2720 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310866676 0 0) 0x7ffb88162de0 con 0x7ffb88071fc0 2024-03-04T12:12:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb74003520 con 0x7ffb88071fc0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924479620 0 0) 0x7ffb881630c0 con 0x7ffb880aead0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb88162de0 con 0x7ffb880aead0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 548866153 0 0) 0x7ffb88162de0 con 0x7ffb880aead0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb880b2720 msgr2=0x7ffb88071a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb880b2720 0x7ffb88071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 >> v1:172.21.15.148:6789/0 conn(0x7ffb88071fc0 legacy=0x7ffb8806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880aeeb0 con 0x7ffb880aead0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7ffb840015f0 con 0x7ffb880aead0 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 >> v1:172.21.15.191:6789/0 conn(0x7ffb880aead0 legacy=0x7ffb88162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb74004ba0 0x7ffb74004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb740037a0 0x7ffb74003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004360 0x7ffb7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb881633a0 con 0x7ffb74004360 2024-03-04T12:12:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400c260 con 0x7ffb740037a0 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400c460 con 0x7ffb74004ba0 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb74004ba0 0x7ffb74004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7ffb8e31b640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004360 0x7ffb7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.422+0000 7f74eeffd640 1 -- 172.21.15.148:0/1571856443 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8005f20 con 0x7f74e800c090 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f800d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f42e8006060 0x7f42e8006f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f800d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80e7b90 con 0x7f42e8048380 2024-03-04T12:12:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f800d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e8007680 con 0x7f42e804fc00 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f800d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e8054ad0 con 0x7f42e8006060 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f5da9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f42e8048380 0x7f42e804f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47370/0 (socket says 172.21.15.148:47370) 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f5da9640 1 -- 172.21.15.148:0/1015704365 learned_addr learned my addr 172.21.15.148:0/1015704365 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7f42f6dab640 1 --2- 172.21.15.148:0/1015704365 >> v2:172.21.15.148:3300/0 conn(0x7f42e8006060 0x7f42e8006f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f42f4da7640 1 -- 172.21.15.148:0/1015704365 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520392290 0 0) 0x7f42e80e7b90 con 0x7f42e8048380 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f42f4da7640 1 -- 172.21.15.148:0/1015704365 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42dc003520 con 0x7f42e8048380 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f42f6dab640 1 -- 172.21.15.148:0/1015704365 >> v1:172.21.15.191:6789/0 conn(0x7f42e804fc00 legacy=0x7f42e80389a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f42f6dab640 1 -- 172.21.15.148:0/1015704365 >> v1:172.21.15.148:6789/0 conn(0x7f42e8048380 legacy=0x7f42e804f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f42f6dab640 1 -- 172.21.15.148:0/1015704365 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8007960 con 0x7f42e8006060 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3394397769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 msgr2=0x7f67e80cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 --2- 172.21.15.148:0/3394397769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 0x7f67e80cfe40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009d90 tx=0x7f67e002be30 comp rx=0 tx=0).stop 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3394397769 shutdown_connections 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 --2- 172.21.15.148:0/3394397769 >> v2:172.21.15.191:3300/0 conn(0x7f67e80c3750 0x7f67e80c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 --2- 172.21.15.148:0/3394397769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cee60 0x7f67e80cfe40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3394397769 >> 172.21.15.148:0/3394397769 conn(0x7f67e80d0780 msgr2=0x7f67e80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3394397769 shutdown_connections 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3394397769 wait complete. 2024-03-04T12:12:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38492/0 (socket says 172.21.15.148:38492) 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.423+0000 7fc07ffff640 1 -- 172.21.15.148:0/3023005882 learned_addr learned my addr 172.21.15.148:0/3023005882 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fc07effd640 1 --2- 172.21.15.148:0/3023005882 >> v2:172.21.15.191:3300/0 conn(0x7fc0800cfd60 0x7fc080162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fc07ffff640 1 -- 172.21.15.148:0/3023005882 >> v2:172.21.15.191:3300/0 conn(0x7fc0800cfd60 msgr2=0x7fc080162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fc07ffff640 1 --2- 172.21.15.148:0/3023005882 >> v2:172.21.15.191:3300/0 conn(0x7fc0800cfd60 0x7fc080162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fc07ffff640 1 -- 172.21.15.148:0/3023005882 >> v1:172.21.15.191:6789/0 conn(0x7fc08015c740 legacy=0x7fc0800cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fc07ffff640 1 -- 172.21.15.148:0/3023005882 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080071540 con 0x7fc080162c40 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27d118640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb27000a200 0x7fb270051cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47374/0 (socket says 172.21.15.148:47374) 2024-03-04T12:12:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.424+0000 7fb27d118640 1 -- 172.21.15.148:0/1629692486 learned_addr learned my addr 172.21.15.148:0/1629692486 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 >> v1:172.21.15.148:6789/0 conn(0x7fa6f40eb060 legacy=0x7fa6f40eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6ec003b40 0x7fa6ec003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7ffb8db1a640 1 -- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb740037a0 msgr2=0x7ffb74003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb740037a0 0x7ffb74003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7ffb8db1a640 1 -- 172.21.15.148:0/951921167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004360 msgr2=0x7ffb7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004360 0x7ffb7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7ffb8db1a640 1 -- 172.21.15.148:0/951921167 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880aeeb0 con 0x7ffb74004ba0 2024-03-04T12:12:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27d919640 1 --2- 172.21.15.148:0/1629692486 >> v2:172.21.15.148:3300/0 conn(0x7fb270009e20 0x7fb270056120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f7871aea640 1 -- 172.21.15.148:0/1226424022 shutdown_connections 2024-03-04T12:12:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 0x7fa6ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec0051c0 0x7fa6ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00bf00 con 0x7fa6ec003b40 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00c0b0 con 0x7fa6ec004800 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00c2b0 con 0x7fa6ec0051c0 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f74eeffd640 1 --2- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e8038070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74e00097a0 tx=0x7f74e002b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352acaf4c43ac3a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 0x7fa6ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa6fbfff640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6ec003b40 0x7fa6ec003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fa700c99640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec0051c0 0x7fa6ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27c917640 1 -- 172.21.15.148:0/1629692486 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141605022 0 0) 0x7fb270056660 con 0x7fb27000a200 2024-03-04T12:12:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27c917640 1 -- 172.21.15.148:0/1629692486 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb264003520 con 0x7fb27000a200 2024-03-04T12:12:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f74edffb640 1 -- 172.21.15.148:0/1571856443 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0037070 con 0x7f74e800c090 2024-03-04T12:12:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f74edffb640 1 -- 172.21.15.148:0/1571856443 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0030710 con 0x7f74e800c090 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f74edffb640 1 -- 172.21.15.148:0/1571856443 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0030ae0 con 0x7f74e800c090 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec0051c0 msgr2=0x7fa6ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec0051c0 0x7fa6ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/2923480286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6ec003b40 msgr2=0x7fa6ec003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6ec003b40 0x7fa6ec003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/2923480286 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec003520 con 0x7fa6ec004800 2024-03-04T12:12:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f7871aea640 1 -- 172.21.15.148:0/1226424022 wait complete. 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7f42f6dab640 1 --2- 172.21.15.148:0/1015704365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e8006060 0x7f42e8006f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0097a0 tx=0x7f42ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f261af320ced6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f42f4da7640 1 -- 172.21.15.148:0/1015704365 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec035020 con 0x7f42e8006060 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fc07ffff640 1 --2- 172.21.15.148:0/3023005882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc078009e50 tx=0x7fc07802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148317f18325d700 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fc07e7fc640 1 -- 172.21.15.148:0/3023005882 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07802ca80 con 0x7fc080162c40 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fc07e7fc640 1 -- 172.21.15.148:0/3023005882 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07802cc20 con 0x7fc080162c40 2024-03-04T12:12:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7fc07e7fc640 1 -- 172.21.15.148:0/3023005882 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780343f0 con 0x7fc080162c40 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f67ed4b1640 1 Processor -- start 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f67ed4b1640 1 -- start start 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f74effff640 1 -- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 msgr2=0x7f74e8038070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f74effff640 1 --2- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e8038070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74e00097a0 tx=0x7f74e002b6e0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27d919640 1 -- 172.21.15.148:0/1629692486 >> v1:172.21.15.191:6789/0 conn(0x7fb27003a230 legacy=0x7fb27003a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27d919640 1 -- 172.21.15.148:0/1629692486 >> v1:172.21.15.148:6789/0 conn(0x7fb27000a200 legacy=0x7fb270051cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.425+0000 7fb27d919640 1 -- 172.21.15.148:0/1629692486 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270055640 con 0x7fb270009e20 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f74effff640 1 -- 172.21.15.148:0/1571856443 shutdown_connections 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f74effff640 1 --2- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8053c60 0x7f74e80df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f74effff640 1 --2- 172.21.15.148:0/1571856443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e8038070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f74effff640 1 -- 172.21.15.148:0/1571856443 >> 172.21.15.148:0/1571856443 conn(0x7f74e8009fa0 msgr2=0x7f74e800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f74effff640 1 -- 172.21.15.148:0/1571856443 shutdown_connections 2024-03-04T12:12:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f74effff640 1 -- 172.21.15.148:0/1571856443 wait complete. 2024-03-04T12:12:19.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fb27d919640 1 --2- 172.21.15.148:0/1629692486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb270009e20 0x7fb270056120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb274009790 tx=0x7fb27402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6058f91f632f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7f42f4da7640 1 -- 172.21.15.148:0/1015704365 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec031830 con 0x7f42e8006060 2024-03-04T12:12:19.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 -- 172.21.15.148:0/1015704365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e8006060 msgr2=0x7f42e8006f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 --2- 172.21.15.148:0/1015704365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e8006060 0x7f42e8006f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0097a0 tx=0x7f42ec02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fa700c99640 1 -- 172.21.15.148:0/2923480286 reap_dead start 2024-03-04T12:12:19.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 -- 172.21.15.148:0/1015704365 shutdown_connections 2024-03-04T12:12:19.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 --2- 172.21.15.148:0/1015704365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e8006060 0x7f42e8006f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 -- 172.21.15.148:0/1015704365 >> 172.21.15.148:0/1015704365 conn(0x7f42e8054c90 msgr2=0x7f42e80550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb74004ba0 0x7ffb74004fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007fd0 tx=0x7ffb74005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab766dc9aa6c95d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84007ca0 con 0x7ffb74004ba0 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84030070 con 0x7ffb74004ba0 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.426+0000 7ffb8cb18640 1 -- 172.21.15.148:0/951921167 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84005d70 con 0x7ffb74004ba0 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fb27c917640 1 -- 172.21.15.148:0/1629692486 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274038070 con 0x7fb270009e20 2024-03-04T12:12:19.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb74004ba0 msgr2=0x7ffb74004fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fb27c917640 1 -- 172.21.15.148:0/1629692486 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274031710 con 0x7fb270009e20 2024-03-04T12:12:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fb27c917640 1 -- 172.21.15.148:0/1629692486 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274031b20 con 0x7fb270009e20 2024-03-04T12:12:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7ffb8fd7e640 1 --2- 172.21.15.148:0/951921167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb74004ba0 0x7ffb74004fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007fd0 tx=0x7ffb74005830 comp rx=0 tx=0).stop 2024-03-04T12:12:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7ffb8e31b640 1 -- 172.21.15.148:0/951921167 reap_dead start 2024-03-04T12:12:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 -- 172.21.15.148:0/3023005882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 msgr2=0x7fc0800c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 --2- 172.21.15.148:0/3023005882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc078009e50 tx=0x7fc07802bee0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 -- 172.21.15.148:0/3023005882 shutdown_connections 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 --2- 172.21.15.148:0/3023005882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc080162c40 0x7fc0800c4470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 --2- 172.21.15.148:0/3023005882 >> v2:172.21.15.191:3300/0 conn(0x7fc0800cfd60 0x7fc080162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 -- 172.21.15.148:0/3023005882 >> 172.21.15.148:0/3023005882 conn(0x7fc08006f5f0 msgr2=0x7fc08006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 shutdown_connections 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 >> 172.21.15.148:0/951921167 conn(0x7ffb8806acd0 msgr2=0x7ffb8806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 -- 172.21.15.148:0/1015704365 shutdown_connections 2024-03-04T12:12:19.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f7871aea640 1 Processor -- start 2024-03-04T12:12:19.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7f42f800d640 1 -- 172.21.15.148:0/1015704365 wait complete. 2024-03-04T12:12:19.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.427+0000 7fc086001640 1 -- 172.21.15.148:0/3023005882 shutdown_connections 2024-03-04T12:12:19.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7fc086001640 1 -- 172.21.15.148:0/3023005882 wait complete. 2024-03-04T12:12:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80c3750 0x7f67e80b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f7871aea640 1 -- start start 2024-03-04T12:12:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e80cee60 0x7f67e80c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 0x7f67e8158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 shutdown_connections 2024-03-04T12:12:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/951921167 wait complete. 2024-03-04T12:12:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806f1d0 con 0x7f67e81621f0 2024-03-04T12:12:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806eef0 con 0x7f67e80cee60 2024-03-04T12:12:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67ed4b1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80afc40 con 0x7f67e80c3750 2024-03-04T12:12:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 0x7fa6ec004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001bc0 tx=0x7fa6ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2d87c651a983b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7fb27e91b640 1 -- 172.21.15.148:0/1629692486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb270009e20 msgr2=0x7fb270056120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7fb27e91b640 1 --2- 172.21.15.148:0/1629692486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb270009e20 0x7fb270056120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb274009790 tx=0x7fb27402b7b0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8031070 con 0x7fa6ec004800 2024-03-04T12:12:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8004aa0 con 0x7fa6ec004800 2024-03-04T12:12:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7fa6faffd640 1 -- 172.21.15.148:0/2923480286 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8001790 con 0x7fa6ec004800 2024-03-04T12:12:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 0x7f67e8158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 0x7f67e8158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39870/0 (socket says 172.21.15.148:39870) 2024-03-04T12:12:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67e77fe640 1 -- 172.21.15.148:0/3369975524 learned_addr learned my addr 172.21.15.148:0/3369975524 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f74effff640 1 Processor -- start 2024-03-04T12:12:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.428+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e80cee60 0x7f67e80c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e6ffd640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80c3750 0x7f67e80b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f74effff640 1 -- start start 2024-03-04T12:12:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f7871aea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 0x7f786c157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 -- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80c3750 msgr2=0x7f67e80b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80c3750 0x7f67e80b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 -- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e80cee60 msgr2=0x7f67e80c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e80cee60 0x7f67e80c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8159300 con 0x7f67e81621f0 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f42f800d640 1 Processor -- start 2024-03-04T12:12:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7fc086001640 1 Processor -- start 2024-03-04T12:12:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f67e77fe640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 0x7f67e8158dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b420 tx=0x7f67dc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f786b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 0x7f786c157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f786b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 0x7f786c157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39872/0 (socket says 172.21.15.148:39872) 2024-03-04T12:12:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f786b7fe640 1 -- 172.21.15.148:0/3774001915 learned_addr learned my addr 172.21.15.148:0/3774001915 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f42f800d640 1 -- start start 2024-03-04T12:12:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.429+0000 7f7871aea640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f786c0b47c0 0x7f786c0a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f7871aea640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c15b960 0x7f786c0a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 -- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 msgr2=0x7fa6ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 0x7fa6ec004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001bc0 tx=0x7fa6ec005830 comp rx=0 tx=0).stop 2024-03-04T12:12:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fb27e91b640 1 -- 172.21.15.148:0/1629692486 shutdown_connections 2024-03-04T12:12:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fb27e91b640 1 --2- 172.21.15.148:0/1629692486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb270009e20 0x7fb270056120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fb27e91b640 1 -- 172.21.15.148:0/1629692486 >> 172.21.15.148:0/1629692486 conn(0x7fb2700537a0 msgr2=0x7fb270053b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7ffb8fd7e640 1 Processor -- start 2024-03-04T12:12:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786affd640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f786c0b47c0 0x7f786c0a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0d0100 con 0x7f786c0b0700 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cf860 con 0x7f786c0b47c0 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c15bd40 con 0x7f786c15b960 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 -- 172.21.15.148:0/2923480286 shutdown_connections 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 --2- 172.21.15.148:0/2923480286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ec004800 0x7fa6ec004c80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 -- 172.21.15.148:0/2923480286 >> 172.21.15.148:0/2923480286 conn(0x7fa6f40559e0 msgr2=0x7fa6f4055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 -- 172.21.15.148:0/2923480286 shutdown_connections 2024-03-04T12:12:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fa70149a640 1 -- 172.21.15.148:0/2923480286 wait complete. 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc011830 con 0x7f67e81621f0 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc016070 con 0x7f67e81621f0 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc011c30 con 0x7f67e81621f0 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fb27e91b640 1 -- 172.21.15.148:0/1629692486 shutdown_connections 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fb27e91b640 1 -- 172.21.15.148:0/1629692486 wait complete. 2024-03-04T12:12:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786b7fe640 1 -- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c15b960 msgr2=0x7f786c0a9470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786b7fe640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f786c15b960 0x7f786c0a9470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786b7fe640 1 -- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f786c0b47c0 msgr2=0x7f786c0a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786b7fe640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f786c0b47c0 0x7f786c0a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f786b7fe640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0cc830 con 0x7f786c0b0700 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f786b7fe640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 0x7f786c157920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c8c0 tx=0x7f785c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fa70149a640 1 Processor -- start 2024-03-04T12:12:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fa70149a640 1 -- start start 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7ffb8fd7e640 1 -- start start 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 0x7ffb881598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb8815b130 0x7ffb8814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f74effff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e80d3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f74effff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 0x7f74e804e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f74effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e8053c60 0x7f74e817c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7f74effff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8037900 con 0x7f74e8053c60 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74effff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80064e0 con 0x7f74e800c090 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74effff640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8037a80 con 0x7f74e8033ee0 2024-03-04T12:12:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 0x7f74e804e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 0x7f74e804e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38526/0 (socket says 172.21.15.148:38526) 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 -- 172.21.15.148:0/1124684324 learned_addr learned my addr 172.21.15.148:0/1124684324 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74eeffd640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e80d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 -- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 msgr2=0x7f74e80d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e800c090 0x7f74e80d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ef7fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e8053c60 0x7f74e817c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 -- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e8053c60 msgr2=0x7f74e817c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e8053c60 0x7f74e817c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ee7fc640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e804c4c0 con 0x7f74e8033ee0 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f74ef7fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e8053c60 0x7f74e817c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f74ee7fc640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 0x7f74e804e400 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c7f0 tx=0x7f74e400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f800d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e8006060 0x7f42e804ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f800d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 0x7f42e80ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f800d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e804fc00 0x7f42e80de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f55a8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 0x7f42e80ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f55a8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 0x7f42e80ddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38538/0 (socket says 172.21.15.148:38538) 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e80c52e0 con 0x7f67e81621f0 2024-03-04T12:12:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f67ed4b1640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e80c5000 con 0x7f67e81621f0 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.430+0000 7fc086001640 1 -- start start 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800cfd60 0x7fc0800b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c740 0x7fc0800c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 0x7fc08015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800cfd60 0x7fc0800b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc07effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c740 0x7fc0800c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 0x7fc08015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 0x7fc08015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39908/0 (socket says 172.21.15.148:39908) 2024-03-04T12:12:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc07ffff640 1 -- 172.21.15.148:0/1906141040 learned_addr learned my addr 172.21.15.148:0/1906141040 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080163270 con 0x7fc080162c40 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801630f0 con 0x7fc08015c740 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800c49b0 con 0x7fc0800cfd60 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814a540 0x7ffb88161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8db1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 0x7ffb881598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8db1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 0x7ffb881598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39910/0 (socket says 172.21.15.148:39910) 2024-03-04T12:12:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8db1a640 1 -- 172.21.15.148:0/2732539466 learned_addr learned my addr 172.21.15.148:0/2732539466 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c017070 con 0x7f786c0b0700 2024-03-04T12:12:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88071c00 con 0x7ffb74004ba0 2024-03-04T12:12:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8e31b640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814a540 0x7ffb88161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88071a80 con 0x7ffb8815b130 2024-03-04T12:12:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c005270 con 0x7f786c0b0700 2024-03-04T12:12:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88071d80 con 0x7ffb8814a540 2024-03-04T12:12:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00a860 con 0x7f786c0b0700 2024-03-04T12:12:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fb27e91b640 1 Processor -- start 2024-03-04T12:12:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fb27e91b640 1 -- start start 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07f7fe640 1 -- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c740 msgr2=0x7fc0800c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07f7fe640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c740 0x7fc0800c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07f7fe640 1 -- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 msgr2=0x7fc08015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07f7fe640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 0x7fc08015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07f7fe640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800c9cc0 con 0x7fc0800cfd60 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7fc07ffff640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080162c40 0x7fc08015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.431+0000 7f42f6dab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e804fc00 0x7f42e80de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f5da9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e8006060 0x7f42e804ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f800d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80381a0 con 0x7f42e804fc00 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f6dab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e804fc00 0x7f42e80de1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39894/0 (socket says 172.21.15.148:39894) 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f6dab640 1 -- 172.21.15.148:0/890504373 learned_addr learned my addr 172.21.15.148:0/890504373 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8038510 con 0x7f42e8006060 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80390b0 con 0x7f42e8048380 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 -- 172.21.15.148:0/2732539466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814a540 msgr2=0x7ffb88161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814a540 0x7ffb88161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 -- 172.21.15.148:0/2732539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb8815b130 msgr2=0x7ffb8814a000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb8815b130 0x7ffb8814a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88159df0 con 0x7ffb74004ba0 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec004800 0x7fa6f4009d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 0x7fa6f40512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f40517e0 0x7fa6f4180910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4057350 con 0x7fa6f40517e0 2024-03-04T12:12:19.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40579a0 con 0x7fa6f40e0f00 2024-03-04T12:12:19.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7fa70149a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4057630 con 0x7fa6ec004800 2024-03-04T12:12:19.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7ffb8db1a640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 0x7ffb881598b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c2a0 tx=0x7ffb840018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.432+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c0cba70 con 0x7f786c0b0700 2024-03-04T12:12:19.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0cc030 con 0x7f786c0b0700 2024-03-04T12:12:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f42f55a8640 1 -- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e8006060 msgr2=0x7f42e804ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f67dc01a410 con 0x7f67e81621f0 2024-03-04T12:12:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f786c0cb820 con 0x7f786c0b0700 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f42f55a8640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e8006060 0x7f42e804ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f42f55a8640 1 -- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e804fc00 msgr2=0x7f42e80de1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f42f55a8640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e804fc00 0x7f42e80de1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.433+0000 7f42f55a8640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8049bf0 con 0x7f42e8048380 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4015070 con 0x7f74e8033ee0 2024-03-04T12:12:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fc07f7fe640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800cfd60 0x7fc0800b2410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c410 tx=0x7fc07000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec004800 0x7fa6f4009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4004220 con 0x7f74e8033ee0 2024-03-04T12:12:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007620 con 0x7f74e8033ee0 2024-03-04T12:12:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e804c760 con 0x7f74e8033ee0 2024-03-04T12:12:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e804cd20 con 0x7f74e8033ee0 2024-03-04T12:12:19.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42f55a8640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 0x7f42e80ddcb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b410 tx=0x7f42e000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070017070 con 0x7fc0800cfd60 2024-03-04T12:12:19.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070005230 con 0x7fc0800cfd60 2024-03-04T12:12:19.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb270009e20 0x7fb27004c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 0x7fb2700e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27003a230 0x7fb2701795a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27000a660 con 0x7fb27003a230 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270052790 con 0x7fb27000a200 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e91b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270052400 con 0x7fb270009e20 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84004bf0 con 0x7ffb74004ba0 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27d118640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 0x7fb2700e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0015020 con 0x7f42e8048380 2024-03-04T12:12:19.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0011830 con 0x7f42e8048380 2024-03-04T12:12:19.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0011c00 con 0x7f42e8048380 2024-03-04T12:12:19.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27d118640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 0x7fb2700e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38066/0 (socket says 172.21.15.148:38066) 2024-03-04T12:12:19.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e8049ed0 con 0x7f42e8048380 2024-03-04T12:12:19.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27e11a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27003a230 0x7fb2701795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27d118640 1 -- 172.21.15.148:0/3594066054 learned_addr learned my addr 172.21.15.148:0/3594066054 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fb27d919640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb270009e20 0x7fb27004c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fa6fb7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 0x7fa6f40512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fa6fb7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 0x7fa6f40512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38060/0 (socket says 172.21.15.148:38060) 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/3738165076 learned_addr learned my addr 172.21.15.148:0/3738165076 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fa700c99640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f40517e0 0x7fa6f4180910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e804a490 con 0x7f42e8048380 2024-03-04T12:12:19.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.434+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700045e0 con 0x7fc0800cfd60 2024-03-04T12:12:19.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84044070 con 0x7ffb74004ba0 2024-03-04T12:12:19.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fb27d118640 1 -- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb270009e20 msgr2=0x7fb27004c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/3738165076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec004800 msgr2=0x7fa6f4009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6ec004800 0x7fa6f4009d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f40517e0 msgr2=0x7fa6f4180910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f40517e0 0x7fa6f4180910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fa6fb7fe640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4181790 con 0x7fa6f40e0f00 2024-03-04T12:12:19.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74e8005380 con 0x7f74e8033ee0 2024-03-04T12:12:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42e8005560 con 0x7f42e8048380 2024-03-04T12:12:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f785c01a070 con 0x7f786c0b0700 2024-03-04T12:12:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.435+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0800c9fa0 con 0x7fc0800cfd60 2024-03-04T12:12:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0800ca560 con 0x7fc0800cfd60 2024-03-04T12:12:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67c0001030 con 0x7f67e81621f0 2024-03-04T12:12:19.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84007c80 con 0x7ffb74004ba0 2024-03-04T12:12:19.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fa6fb7fe640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 0x7fa6f40512a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000fb0 tx=0x7fa6e8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8815a090 con 0x7ffb74004ba0 2024-03-04T12:12:19.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8815a650 con 0x7ffb74004ba0 2024-03-04T12:12:19.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f74e40077c0 con 0x7f74e8033ee0 2024-03-04T12:12:19.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fb27d118640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb270009e20 0x7fb27004c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fb27d118640 1 -- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27003a230 msgr2=0x7fb2701795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fb27d118640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27003a230 0x7fb2701795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7f7853fff640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f785806b5a0 0x7f785806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.436+0000 7fb27d118640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27004c800 con 0x7fb27000a200 2024-03-04T12:12:19.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb27e11a640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27003a230 0x7fb2701795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:19.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f786affd640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f785806b5a0 0x7f785806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74d77fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f74dc06b520 0x7f74dc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc080074f60 con 0x7fc0800cfd60 2024-03-04T12:12:19.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb27d118640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 0x7fb2700e0660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c500 tx=0x7fb26800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f42e0018020 con 0x7f42e8048380 2024-03-04T12:12:19.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f74e408a240 con 0x7f74e8033ee0 2024-03-04T12:12:19.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42d406b5d0 0x7f42d406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb881481f0 con 0x7ffb74004ba0 2024-03-04T12:12:19.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74d77fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] conn(0x7f74dc074ca0 0x7f74dc0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f42e008a9f0 con 0x7f42e8048380 2024-03-04T12:12:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42f5da9640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42d406b5d0 0x7f42d406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74dc0777b0 con 0x7f74dc074ca0 2024-03-04T12:12:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] conn(0x7f42d4074d50 0x7f42d4077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e4091050 con 0x7f74e8033ee0 2024-03-04T12:12:19.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74eeffd640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f74dc06b520 0x7f74dc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42e000fe50 con 0x7f42d4074d50 2024-03-04T12:12:19.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e008ade0 con 0x7f42e8048380 2024-03-04T12:12:19.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb27000a200 2024-03-04T12:12:19.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7f74ef7fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] conn(0x7f74dc074ca0 0x7f74dc0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f786affd640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f785806b5a0 0x7f785806da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7864035000 tx=0x7f7864030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268005ce0 con 0x7fb27000a200 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8031070 con 0x7fa6f40e0f00 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8032040 con 0x7fa6f40e0f00 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e802d700 con 0x7fa6f40e0f00 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680072f0 con 0x7fb27000a200 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27017b9b0 con 0x7fb27000a200 2024-03-04T12:12:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f785c08a510 con 0x7f786c0b0700 2024-03-04T12:12:19.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f42f5da9640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42d406b5d0 0x7f42d406da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000b60 tx=0x7f42f008af90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.437+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27017bf70 con 0x7fb27000a200 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f7853fff640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] conn(0x7f7858074ca0 0x7f78580770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78580777b0 con 0x7f7858074ca0 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c08a900 con 0x7f786c0b0700 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f786bfff640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] conn(0x7f7858074ca0 0x7f78580770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc07001a070 con 0x7fc0800cfd60 2024-03-04T12:12:19.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7f786bfff640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] conn(0x7f7858074ca0 0x7f78580770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f40504e0 con 0x7fa6f40e0f00 2024-03-04T12:12:19.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f4050aa0 con 0x7fa6f40e0f00 2024-03-04T12:12:19.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fc067fff640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc06c06b5a0 0x7fc06c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc070089710 con 0x7fc0800cfd60 2024-03-04T12:12:19.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7f42f6dab640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] conn(0x7f42d4074d50 0x7f42d4077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2700057d0 con 0x7fb27000a200 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67cf7fe640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67d406b5d0 0x7f67d406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f67dc089b80 con 0x7f67e81621f0 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67cf7fe640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] conn(0x7f67d4074d50 0x7f67d4077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67d4077860 con 0x7f67d4074d50 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67dc090050 con 0x7f67e81621f0 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.438+0000 7f67e6ffd640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67d406b5d0 0x7f67d406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] conn(0x7f67d4074d50 0x7f67d4077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fc067fff640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] conn(0x7fc06c074d10 0x7fc06c077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fc07effd640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc06c06b5a0 0x7fc06c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7f67e67fc640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] conn(0x7f67d4074d50 0x7f67d4077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc06c077820 con 0x7fc06c074d10 2024-03-04T12:12:19.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.439+0000 7f74ef7fe640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] conn(0x7f74dc074ca0 0x7f74dc0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fc07ffff640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] conn(0x7fc06c074d10 0x7fc06c077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc070090050 con 0x7fc0800cfd60 2024-03-04T12:12:19.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6f40057e0 con 0x7fa6f40e0f00 2024-03-04T12:12:19.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f42f6dab640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] conn(0x7f42d4074d50 0x7f42d4077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffb84007070 con 0x7ffb74004ba0 2024-03-04T12:12:19.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f67e6ffd640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67d406b5d0 0x7f67d406da90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67d80012b0 tx=0x7f67d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fc07effd640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc06c06b5a0 0x7fc06c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc080164280 tx=0x7fc074008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f74eeffd640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f74dc06b520 0x7f74dc06d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74e0034000 tx=0x7f74e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== osd.1 v2:172.21.15.148:6824/3160501011 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f78580777b0 con 0x7f7858074ca0 2024-03-04T12:12:19.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== osd.6 v2:172.21.15.191:6808/1171226865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f67d4077860 con 0x7f67d4074d50 2024-03-04T12:12:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fc07ffff640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] conn(0x7fc06c074d10 0x7fc06c077150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7ffb7e7fc640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffb6c06b520 0x7ffb6c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffb840ac990 con 0x7ffb74004ba0 2024-03-04T12:12:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb26801a070 con 0x7fb27000a200 2024-03-04T12:12:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fb26e7fc640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa6e802d8a0 con 0x7fa6f40e0f00 2024-03-04T12:12:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== osd.5 v2:172.21.15.191:6816/2924287970 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42e000fe50 con 0x7f42d4074d50 2024-03-04T12:12:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fb27d919640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7ffb7e7fc640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] conn(0x7ffb6c074ca0 0x7ffb6c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb8400bf30 con 0x7ffb6c074ca0 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb840accc0 con 0x7ffb74004ba0 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.440+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== osd.4 v2:172.21.15.191:6800/2064455370 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f74dc0777b0 con 0x7f74dc074ca0 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6f8ff9640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa6e006b5a0 0x7fa6e006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa6e80ac530 con 0x7fa6f40e0f00 2024-03-04T12:12:19.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7ffb8d319640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffb6c06b520 0x7ffb6c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6fbfff640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa6e006b5a0 0x7fa6e006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== osd.7 v2:172.21.15.191:6824/1686676970 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc06c077820 con 0x7fc06c074d10 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6f8ff9640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] conn(0x7fa6e0074d20 0x7fa6e0077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6e803ae50 con 0x7fa6e0074d20 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.441+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e80b3050 con 0x7fa6f40e0f00 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fa700c99640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] conn(0x7fa6e0074d20 0x7fa6e0077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb268055e10 con 0x7fb27000a200 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fb26e7fc640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] conn(0x7fb25c075820 0x7fb25c077cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb25c078390 con 0x7fb25c075820 2024-03-04T12:12:19.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fa700c99640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] conn(0x7fa6e0074d20 0x7fa6e0077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fb27e11a640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] conn(0x7fb25c075820 0x7fb25c077cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7fa6fbfff640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa6e006b5a0 0x7fa6e006da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0683d0 tx=0x7fa6fc08af90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.442+0000 7ffb8e31b640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] conn(0x7ffb6c074ca0 0x7ffb6c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.443+0000 7fb27e11a640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] conn(0x7fb25c075820 0x7fb25c077cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.443+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== osd.2 v2:172.21.15.148:6808/2881995606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa6e803ae50 con 0x7fa6e0074d20 2024-03-04T12:12:19.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.444+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== osd.0 v2:172.21.15.148:6800/3066481003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb25c078390 con 0x7fb25c075820 2024-03-04T12:12:19.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.443+0000 7ffb8e31b640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] conn(0x7ffb6c074ca0 0x7ffb6c0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.444+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== osd.3 v2:172.21.15.148:6809/3796620207 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffb8400bf30 con 0x7ffb6c074ca0 2024-03-04T12:12:19.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.444+0000 7fb27d919640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb274035000 tx=0x7fb274008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.445+0000 7ffb8d319640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffb6c06b520 0x7ffb6c06d9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb8814b660 tx=0x7ffb78006b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.445+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2680547a0 con 0x7fb27000a200 2024-03-04T12:12:19.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.459+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 --> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb88161ad0 con 0x7ffb6c074ca0 2024-03-04T12:12:19.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.460+0000 7ffb7e7fc640 1 -- 172.21.15.148:0/2732539466 <== osd.3 v2:172.21.15.148:6809/3796620207 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb88161ad0 con 0x7ffb6c074ca0 2024-03-04T12:12:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.460+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] conn(0x7ffb6c074ca0 msgr2=0x7ffb6c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.460+0000 7ffb8fd7e640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6809/3796620207,v1:172.21.15.148:6811/3796620207] conn(0x7ffb6c074ca0 0x7ffb6c0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffb6c06b520 msgr2=0x7ffb6c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffb6c06b520 0x7ffb6c06d9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb8814b660 tx=0x7ffb78006b30 comp rx=0 tx=0).stop 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 msgr2=0x7ffb881598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 --2- 172.21.15.148:0/2732539466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74004ba0 0x7ffb881598b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c2a0 tx=0x7ffb840018f0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8e31b640 1 -- 172.21.15.148:0/2732539466 reap_dead start 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 shutdown_connections 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.461+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 >> 172.21.15.148:0/2732539466 conn(0x7ffb8806acd0 msgr2=0x7ffb880b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.462+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 shutdown_connections 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.462+0000 7ffb8fd7e640 1 -- 172.21.15.148:0/2732539466 wait complete. 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.472+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 --> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f786c074f60 con 0x7f7858074ca0 2024-03-04T12:12:19.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.473+0000 7f7853fff640 1 -- 172.21.15.148:0/3774001915 <== osd.1 v2:172.21.15.148:6824/3160501011 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f786c074f60 con 0x7f7858074ca0 2024-03-04T12:12:19.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.473+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 --> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67c0002dd0 con 0x7f67d4074d50 2024-03-04T12:12:19.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.473+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] conn(0x7f7858074ca0 msgr2=0x7f78580770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.473+0000 7f7871aea640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6824/3160501011,v1:172.21.15.148:6825/3160501011] conn(0x7f7858074ca0 0x7f78580770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.473+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 --> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0800b0d50 con 0x7fc06c074d10 2024-03-04T12:12:19.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f785806b5a0 msgr2=0x7f785806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f785806b5a0 0x7f785806da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7864035000 tx=0x7f7864030000 comp rx=0 tx=0).stop 2024-03-04T12:12:19.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 msgr2=0x7f786c157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 --2- 172.21.15.148:0/3774001915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c0b0700 0x7f786c157920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c8c0 tx=0x7f785c00cd80 comp rx=0 tx=0).stop 2024-03-04T12:12:19.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f786bfff640 1 -- 172.21.15.148:0/3774001915 reap_dead start 2024-03-04T12:12:19.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f67cf7fe640 1 -- 172.21.15.148:0/3369975524 <== osd.6 v2:172.21.15.191:6808/1171226865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67c0002dd0 con 0x7f67d4074d50 2024-03-04T12:12:19.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7fc067fff640 1 -- 172.21.15.148:0/1906141040 <== osd.7 v2:172.21.15.191:6824/1686676970 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0800b0d50 con 0x7fc06c074d10 2024-03-04T12:12:19.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 shutdown_connections 2024-03-04T12:12:19.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 >> 172.21.15.148:0/3774001915 conn(0x7f786c06f5f0 msgr2=0x7f786c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 shutdown_connections 2024-03-04T12:12:19.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7f7871aea640 1 -- 172.21.15.148:0/3774001915 wait complete. 2024-03-04T12:12:19.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] conn(0x7fc06c074d10 msgr2=0x7fc06c077150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.474+0000 7fc086001640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:6824/1686676970,v1:172.21.15.191:6825/1686676970] conn(0x7fc06c074d10 0x7fc06c077150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 --> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6f404c7e0 con 0x7fa6e0074d20 2024-03-04T12:12:19.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] conn(0x7f67d4074d50 msgr2=0x7f67d4077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.191:6808/1171226865,v1:172.21.15.191:6809/1171226865] conn(0x7f67d4074d50 0x7f67d4077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc06c06b5a0 msgr2=0x7fc06c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc06c06b5a0 0x7fc06c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc080164280 tx=0x7fc074008040 comp rx=0 tx=0).stop 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800cfd60 msgr2=0x7fc0800b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 --2- 172.21.15.148:0/1906141040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800cfd60 0x7fc0800b2410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c410 tx=0x7fc07000c8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc07ffff640 1 -- 172.21.15.148:0/1906141040 reap_dead start 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 shutdown_connections 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 >> 172.21.15.148:0/1906141040 conn(0x7fc08006f5f0 msgr2=0x7fc080164eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67d406b5d0 msgr2=0x7f67d406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67d406b5d0 0x7f67d406da90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67d80012b0 tx=0x7f67d8009040 comp rx=0 tx=0).stop 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 msgr2=0x7f67e8158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 --2- 172.21.15.148:0/3369975524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67e81621f0 0x7f67e8158dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b420 tx=0x7f67dc00b8e0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67e77fe640 1 -- 172.21.15.148:0/3369975524 reap_dead start 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 shutdown_connections 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 >> 172.21.15.148:0/3369975524 conn(0x7f67e80d0780 msgr2=0x7f67e80b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.475+0000 7fa6f8ff9640 1 -- 172.21.15.148:0/3738165076 <== osd.2 v2:172.21.15.148:6808/2881995606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa6f404c7e0 con 0x7fa6e0074d20 2024-03-04T12:12:19.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 shutdown_connections 2024-03-04T12:12:19.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] conn(0x7fa6e0074d20 msgr2=0x7fa6e0077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6808/2881995606,v1:172.21.15.148:6810/2881995606] conn(0x7fa6e0074d20 0x7fa6e0077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa6e006b5a0 msgr2=0x7fa6e006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa6e006b5a0 0x7fa6e006da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0683d0 tx=0x7fa6fc08af90 comp rx=0 tx=0).stop 2024-03-04T12:12:19.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fc086001640 1 -- 172.21.15.148:0/1906141040 wait complete. 2024-03-04T12:12:19.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 msgr2=0x7fa6f40512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 --2- 172.21.15.148:0/3738165076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f40e0f00 0x7fa6f40512a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000fb0 tx=0x7fa6e8000bc0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa700c99640 1 -- 172.21.15.148:0/3738165076 reap_dead start 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 shutdown_connections 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 >> 172.21.15.148:0/3738165076 conn(0x7fa6f40559e0 msgr2=0x7fa6f40e8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 shutdown_connections 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7fa70149a640 1 -- 172.21.15.148:0/3738165076 wait complete. 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.476+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 shutdown_connections 2024-03-04T12:12:19.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.477+0000 7f67cd7fa640 1 -- 172.21.15.148:0/3369975524 wait complete. 2024-03-04T12:12:19.533 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.477+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 --> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42e803aac0 con 0x7f42d4074d50 2024-03-04T12:12:19.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T12:12:19.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.477+0000 7f42e67fc640 1 -- 172.21.15.148:0/890504373 <== osd.5 v2:172.21.15.191:6816/2924287970 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f42e803aac0 con 0x7f42d4074d50 2024-03-04T12:12:19.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] conn(0x7f42d4074d50 msgr2=0x7f42d4077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:6816/2924287970,v1:172.21.15.191:6817/2924287970] conn(0x7f42d4074d50 0x7f42d4077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42d406b5d0 msgr2=0x7f42d406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42d406b5d0 0x7f42d406da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000b60 tx=0x7f42f008af90 comp rx=0 tx=0).stop 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 msgr2=0x7f42e80ddcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f800d640 1 --2- 172.21.15.148:0/890504373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8048380 0x7f42e80ddcb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b410 tx=0x7f42e000b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.478+0000 7f42f6dab640 1 -- 172.21.15.148:0/890504373 reap_dead start 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.479+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 shutdown_connections 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.479+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 >> 172.21.15.148:0/890504373 conn(0x7f42e8054c90 msgr2=0x7f42e80e8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.479+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 shutdown_connections 2024-03-04T12:12:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.479+0000 7f42f800d640 1 -- 172.21.15.148:0/890504373 wait complete. 2024-03-04T12:12:19.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.479+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 --> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f74e8052bd0 con 0x7f74dc074ca0 2024-03-04T12:12:19.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.480+0000 7f74d77fe640 1 -- 172.21.15.148:0/1124684324 <== osd.4 v2:172.21.15.191:6800/2064455370 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f74e8052bd0 con 0x7f74dc074ca0 2024-03-04T12:12:19.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] conn(0x7f74dc074ca0 msgr2=0x7f74dc0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:6800/2064455370,v1:172.21.15.191:6801/2064455370] conn(0x7f74dc074ca0 0x7f74dc0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f74dc06b520 msgr2=0x7f74dc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f74dc06b520 0x7f74dc06d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74e0034000 tx=0x7f74e0008030 comp rx=0 tx=0).stop 2024-03-04T12:12:19.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 msgr2=0x7f74e804e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.481+0000 7f74effff640 1 --2- 172.21.15.148:0/1124684324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74e8033ee0 0x7f74e804e400 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c7f0 tx=0x7f74e400ccb0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7f74ef7fe640 1 -- 172.21.15.148:0/1124684324 reap_dead start 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 shutdown_connections 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 >> 172.21.15.148:0/1124684324 conn(0x7f74e8009fa0 msgr2=0x7f74e80358c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 shutdown_connections 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7f74effff640 1 -- 172.21.15.148:0/1124684324 wait complete. 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.482+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 --> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb270054530 con 0x7fb25c075820 2024-03-04T12:12:19.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.483+0000 7fb26e7fc640 1 -- 172.21.15.148:0/3594066054 <== osd.0 v2:172.21.15.148:6800/3066481003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb270054530 con 0x7fb25c075820 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.484+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] conn(0x7fb25c075820 msgr2=0x7fb25c077cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.484+0000 7fb27e91b640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6800/3066481003,v1:172.21.15.148:6801/3066481003] conn(0x7fb25c075820 0x7fb25c077cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.485+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb25c06b4d0 msgr2=0x7fb25c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.485+0000 7fb27e91b640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb274035000 tx=0x7fb274008030 comp rx=0 tx=0).stop 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.485+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 msgr2=0x7fb2700e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.485+0000 7fb27e91b640 1 --2- 172.21.15.148:0/3594066054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27000a200 0x7fb2700e0660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c500 tx=0x7fb26800c9c0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.485+0000 7fb27e11a640 1 -- 172.21.15.148:0/3594066054 reap_dead start 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.487+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 shutdown_connections 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.487+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 >> 172.21.15.148:0/3594066054 conn(0x7fb2700537a0 msgr2=0x7fb270009320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.487+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 shutdown_connections 2024-03-04T12:12:19.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.487+0000 7fb27e91b640 1 -- 172.21.15.148:0/3594066054 wait complete. 2024-03-04T12:12:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T12:12:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:19.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T12:12:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T12:12:19.545 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T12:12:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:19.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T12:12:19.547 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T12:12:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:19.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T12:12:19.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.748+0000 7f1dd7fc6640 1 Processor -- start 2024-03-04T12:12:19.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.748+0000 7f182ea20640 1 Processor -- start 2024-03-04T12:12:19.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.748+0000 7f182ea20640 1 -- start start 2024-03-04T12:12:19.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.748+0000 7f1dd7fc6640 1 -- start start 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7f1dd7fc6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dd015ce00 0x7f1dd00b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7f1dd7fc6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015da60 0x7f1dd0099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7f1dd7fc6640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd009b070 con 0x7f1dd015d5f0 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7f1dd7fc6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd00cca10 con 0x7f1dd015ce00 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7f1dd7fc6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd015d1e0 con 0x7f1dd015da60 2024-03-04T12:12:19.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7fdae8a81640 1 Processor -- start 2024-03-04T12:12:19.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.749+0000 7fdae8a81640 1 -- start start 2024-03-04T12:12:19.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f1dd5561640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1dd015d5f0 0x7f1dd00ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58654/0 (socket says 172.21.15.148:58654) 2024-03-04T12:12:19.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f18200df210 0x7f1820038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1820053c60 0x7f182004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f1dd5561640 1 -- 172.21.15.148:0/3796349769 learned_addr learned my addr 172.21.15.148:0/3796349769 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820033ee0 0x7f182004ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200544b0 con 0x7f1820053c60 2024-03-04T12:12:19.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200541d0 con 0x7f18200df210 2024-03-04T12:12:19.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f182ea20640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1820005f20 con 0x7f1820033ee0 2024-03-04T12:12:19.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f1dd5d62640 1 --2- 172.21.15.148:0/3796349769 >> v2:172.21.15.148:3300/0 conn(0x7f1dd015ce00 0x7f1dd00b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f18277fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1820053c60 0x7f182004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f18277fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1820053c60 0x7f182004fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39958/0 (socket says 172.21.15.148:39958) 2024-03-04T12:12:19.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae8a81640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdadc0df030 0x7fdadc038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae8a81640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f18277fe640 1 -- 172.21.15.148:0/1424957851 learned_addr learned my addr 172.21.15.148:0/1424957851 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae8a81640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc005f60 con 0x7fdadc0df410 2024-03-04T12:12:19.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae8a81640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc0e38e0 con 0x7fdadc0df030 2024-03-04T12:12:19.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae8a81640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc006200 con 0x7fdadc050140 2024-03-04T12:12:19.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f1dd5d62640 1 -- 172.21.15.148:0/3796349769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015da60 msgr2=0x7f1dd0099a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:19.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f334a48c640 1 Processor -- start 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f1dd5d62640 1 --2- 172.21.15.148:0/3796349769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015da60 0x7f1dd0099a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f1dd5d62640 1 -- 172.21.15.148:0/3796349769 >> v1:172.21.15.191:6789/0 conn(0x7f1dd015d5f0 legacy=0x7f1dd00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f1dd5d62640 1 -- 172.21.15.148:0/3796349769 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00cffa0 con 0x7f1dd015ce00 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38592/0 (socket says 172.21.15.148:38592) 2024-03-04T12:12:19.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 -- 172.21.15.148:0/3169181878 learned_addr learned my addr 172.21.15.148:0/3169181878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f182cfbd640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820033ee0 0x7f182004ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f18277fe640 1 -- 172.21.15.148:0/1424957851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820033ee0 msgr2=0x7f182004ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f18277fe640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820033ee0 0x7f182004ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f18277fe640 1 -- 172.21.15.148:0/1424957851 >> v2:172.21.15.191:3300/0 conn(0x7f18200df210 msgr2=0x7f1820038070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f18277fe640 1 --2- 172.21.15.148:0/1424957851 >> v2:172.21.15.191:3300/0 conn(0x7f18200df210 0x7f1820038070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f18277fe640 1 -- 172.21.15.148:0/1424957851 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820006200 con 0x7f1820053c60 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.750+0000 7f334a48c640 1 -- start start 2024-03-04T12:12:19.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f334a48c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f33440cef00 0x7f334415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f334a48c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33440ce180 con 0x7f334415b7c0 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f334a48c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334415ef10 con 0x7f33440d0a20 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7f334a48c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33440d3690 con 0x7f33440cef00 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f29dbdf1640 1 Processor -- start 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f33437fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f33440d0a20 0x7f334415e1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58666/0 (socket says 172.21.15.148:58666) 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f33437fe640 1 -- 172.21.15.148:0/4201017439 learned_addr learned my addr 172.21.15.148:0/4201017439 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f3343fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f334415b7c0 0x7f33440d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47388/0 (socket says 172.21.15.148:47388) 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f3348a29640 1 --2- 172.21.15.148:0/4201017439 >> v2:172.21.15.148:3300/0 conn(0x7f33440cef00 0x7f334415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f3348a29640 1 -- 172.21.15.148:0/4201017439 >> v1:172.21.15.191:6789/0 conn(0x7f33440d0a20 legacy=0x7f334415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f3348a29640 1 -- 172.21.15.148:0/4201017439 >> v1:172.21.15.148:6789/0 conn(0x7f334415b7c0 legacy=0x7f33440d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f3348a29640 1 -- 172.21.15.148:0/4201017439 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440ce420 con 0x7f33440cef00 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.752+0000 7f1dd5d62640 1 --2- 172.21.15.148:0/3796349769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015ce00 0x7f1dd00b0610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009d90 tx=0x7f1dc802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2550fbe33a50af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 -- 172.21.15.148:0/3169181878 >> v2:172.21.15.148:3300/0 conn(0x7fdadc0df030 msgr2=0x7fdadc038170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/3169181878 >> v2:172.21.15.148:3300/0 conn(0x7fdadc0df030 0x7fdadc038170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 -- 172.21.15.148:0/3169181878 >> v1:172.21.15.148:6789/0 conn(0x7fdadc0df410 legacy=0x7fdadc04fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:19.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.751+0000 7fdae2d9d640 1 -- 172.21.15.148:0/3169181878 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0064e0 con 0x7fdadc050140 2024-03-04T12:12:19.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f1dd4d60640 1 -- 172.21.15.148:0/3796349769 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8039070 con 0x7f1dd015ce00 2024-03-04T12:12:19.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f29dbdf1640 1 -- start start 2024-03-04T12:12:19.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f18277fe640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820053c60 0x7f182004fa80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f182000b090 tx=0x7f181402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3a0446eef7ebe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f1826ffd640 1 -- 172.21.15.148:0/1424957851 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814035020 con 0x7f1820053c60 2024-03-04T12:12:19.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f3348a29640 1 --2- 172.21.15.148:0/4201017439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440cef00 0x7f334415e8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f333c009d90 tx=0x7f333c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b1953e30b5093 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f1826ffd640 1 -- 172.21.15.148:0/1424957851 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814031810 con 0x7f1820053c60 2024-03-04T12:12:19.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/3169181878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdae4000b60 tx=0x7fdae4068aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2313d7f94b6b8ea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.753+0000 7f1dd4d60640 1 -- 172.21.15.148:0/3796349769 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc802cbf0 con 0x7f1dd015ce00 2024-03-04T12:12:19.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f3342ffd640 1 -- 172.21.15.148:0/4201017439 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c039070 con 0x7f33440cef00 2024-03-04T12:12:19.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7ff0c63e1640 1 Processor -- start 2024-03-04T12:12:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f3342ffd640 1 -- 172.21.15.148:0/4201017439 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c02ccf0 con 0x7f33440cef00 2024-03-04T12:12:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7ff0c63e1640 1 -- start start 2024-03-04T12:12:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/3796349769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015ce00 msgr2=0x7f1dd00b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/3796349769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015ce00 0x7f1dd00b0610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009d90 tx=0x7f1dc802be40 comp rx=0 tx=0).stop 2024-03-04T12:12:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/3796349769 shutdown_connections 2024-03-04T12:12:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/3796349769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015da60 0x7f1dd0099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/3796349769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015ce00 0x7f1dd00b0610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/3796349769 >> 172.21.15.148:0/3796349769 conn(0x7f1dd00b3180 msgr2=0x7f1dd00b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f29dbdf1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29cc0382a0 0x7f29cc058580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f29d9b8d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f29cc0e3390 0x7f29cc056d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47392/0 (socket says 172.21.15.148:47392) 2024-03-04T12:12:19.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/3796349769 shutdown_connections 2024-03-04T12:12:19.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/3796349769 wait complete. 2024-03-04T12:12:19.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7fdae159a640 1 -- 172.21.15.148:0/3169181878 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40af070 con 0x7fdadc050140 2024-03-04T12:12:19.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7fdae159a640 1 -- 172.21.15.148:0/3169181878 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40a9c80 con 0x7fdadc050140 2024-03-04T12:12:19.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7fdae159a640 1 -- 172.21.15.148:0/3169181878 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae406ce60 con 0x7fdadc050140 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f334a48c640 1 -- 172.21.15.148:0/4201017439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440cef00 msgr2=0x7f334415e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f334a48c640 1 --2- 172.21.15.148:0/4201017439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440cef00 0x7f334415e8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f333c009d90 tx=0x7f333c02bf40 comp rx=0 tx=0).stop 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f334a48c640 1 -- 172.21.15.148:0/4201017439 shutdown_connections 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f334a48c640 1 --2- 172.21.15.148:0/4201017439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440cef00 0x7f334415e8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f334a48c640 1 -- 172.21.15.148:0/4201017439 >> 172.21.15.148:0/4201017439 conn(0x7f3344072400 msgr2=0x7f3344072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.754+0000 7f29d9b8d640 1 -- 172.21.15.148:0/1190150886 learned_addr learned my addr 172.21.15.148:0/1190150886 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc0574e0 con 0x7f29cc0e3390 2024-03-04T12:12:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc0577c0 con 0x7f29cc058ac0 2024-03-04T12:12:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7fdae8a81640 1 -- 172.21.15.148:0/3169181878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 msgr2=0x7fdadc006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7fdae8a81640 1 --2- 172.21.15.148:0/3169181878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdae4000b60 tx=0x7fdae4068aa0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc054780 con 0x7f29cc0382a0 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7fdae8a81640 1 -- 172.21.15.148:0/3169181878 shutdown_connections 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7ff0c63e1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff0c0161210 0x7ff0c0162390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0c63e1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0c00b5ca0 0x7ff0c00d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0c63e1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00ce370 con 0x7ff0c00b2f30 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0c63e1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c01628d0 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0c63e1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00ce650 con 0x7ff0c0161210 2024-03-04T12:12:19.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0c00b2f30 0x7ff0c00b5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47396/0 (socket says 172.21.15.148:47396) 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bf7fe640 1 -- 172.21.15.148:0/4289962696 learned_addr learned my addr 172.21.15.148:0/4289962696 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 -- 172.21.15.148:0/1424957851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820053c60 msgr2=0x7f182004fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820053c60 0x7f182004fa80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f182000b090 tx=0x7f181402b8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 -- 172.21.15.148:0/1424957851 shutdown_connections 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820033ee0 0x7f182004ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 --2- 172.21.15.148:0/1424957851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820053c60 0x7f182004fa80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 --2- 172.21.15.148:0/1424957851 >> v2:172.21.15.191:3300/0 conn(0x7f18200df210 0x7f1820038070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f182ea20640 1 -- 172.21.15.148:0/1424957851 >> 172.21.15.148:0/1424957851 conn(0x7f1820009fa0 msgr2=0x7f182000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f334a48c640 1 -- 172.21.15.148:0/4201017439 shutdown_connections 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f334a48c640 1 -- 172.21.15.148:0/4201017439 wait complete. 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7fdae8a81640 1 --2- 172.21.15.148:0/3169181878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc050140 0x7fdadc006a00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7fdae8a81640 1 --2- 172.21.15.148:0/3169181878 >> v2:172.21.15.148:3300/0 conn(0x7fdadc0df030 0x7fdadc038170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7fdae8a81640 1 -- 172.21.15.148:0/3169181878 >> 172.21.15.148:0/3169181878 conn(0x7fdadc009fe0 msgr2=0x7fdadc00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bffff640 1 --2- 172.21.15.148:0/4289962696 >> v2:172.21.15.191:3300/0 conn(0x7ff0c0161210 0x7ff0c0162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0c497e640 1 --2- 172.21.15.148:0/4289962696 >> v2:172.21.15.148:3300/0 conn(0x7ff0c00b5ca0 0x7ff0c00d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f182ea20640 1 -- 172.21.15.148:0/1424957851 shutdown_connections 2024-03-04T12:12:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f1dd7fc6640 1 Processor -- start 2024-03-04T12:12:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f11cdb9e640 1 Processor -- start 2024-03-04T12:12:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f11cdb9e640 1 -- start start 2024-03-04T12:12:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.755+0000 7f29d938c640 1 --2- 172.21.15.148:0/1190150886 >> v2:172.21.15.148:3300/0 conn(0x7f29cc0382a0 0x7f29cc058580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f29d938c640 1 -- 172.21.15.148:0/1190150886 >> v1:172.21.15.191:6789/0 conn(0x7f29cc058ac0 legacy=0x7f29cc053990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f29d938c640 1 -- 172.21.15.148:0/1190150886 >> v1:172.21.15.148:6789/0 conn(0x7f29cc0e3390 legacy=0x7f29cc056d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f56dab3c640 1 Processor -- start 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7fdae8a81640 1 -- 172.21.15.148:0/3169181878 shutdown_connections 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f56dab3c640 1 -- start start 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7f1dd7fc6640 1 -- start start 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015ce00 0x7f1dd01572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd015da60 0x7f1dd00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00cc6a0 con 0x7f1dd015d5f0 2024-03-04T12:12:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00d0560 con 0x7f1dd015da60 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f1dd7fc6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00cc350 con 0x7f1dd015ce00 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7fdae8a81640 1 -- 172.21.15.148:0/3169181878 wait complete. 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bffff640 1 -- 172.21.15.148:0/4289962696 >> v2:172.21.15.148:3300/0 conn(0x7ff0c00b5ca0 msgr2=0x7ff0c00d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bffff640 1 --2- 172.21.15.148:0/4289962696 >> v2:172.21.15.148:3300/0 conn(0x7ff0c00b5ca0 0x7ff0c00d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bffff640 1 -- 172.21.15.148:0/4289962696 >> v1:172.21.15.148:6789/0 conn(0x7ff0c00b2f30 legacy=0x7ff0c00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.756+0000 7ff0bffff640 1 -- 172.21.15.148:0/4289962696 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00ce930 con 0x7ff0c0161210 2024-03-04T12:12:19.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f182ea20640 1 -- 172.21.15.148:0/1424957851 wait complete. 2024-03-04T12:12:19.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f29d938c640 1 -- 172.21.15.148:0/1190150886 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc057aa0 con 0x7f29cc0382a0 2024-03-04T12:12:19.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.757+0000 7f334a48c640 1 Processor -- start 2024-03-04T12:12:19.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f11c815aa90 0x7f11c806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40000/0 (socket says 172.21.15.148:40000) 2024-03-04T12:12:19.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f334a48c640 1 -- start start 2024-03-04T12:12:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 -- 172.21.15.148:0/2321532292 learned_addr learned my addr 172.21.15.148:0/2321532292 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f56dab3c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f56cc0e59a0 0x7f56cc0e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 0x7f11c80cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f29d938c640 1 --2- 172.21.15.148:0/1190150886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0382a0 0x7f29cc058580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794bf7636f8f31ed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7ff0bffff640 1 --2- 172.21.15.148:0/4289962696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c0161210 0x7ff0c0162390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00b6530 tx=0x7ff0b802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99f1fb4353ba555 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11c6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c8070600 con 0x7f11c806f470 2024-03-04T12:12:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c806f9f0 con 0x7f11c815aa90 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11cdb9e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806fcd0 con 0x7f11c80cf110 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11c77fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f11c815aa90 0x7f11c806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11c6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40012/0 (socket says 172.21.15.148:40012) 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11c6ffd640 1 -- 172.21.15.148:0/642100258 learned_addr learned my addr 172.21.15.148:0/642100258 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f11c7fff640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 0x7f11c80cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 -- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015ce00 msgr2=0x7f1dd01572a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015ce00 0x7f1dd01572a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 -- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd015da60 msgr2=0x7f1dd00b5890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T12:12:19.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd015da60 0x7f1dd00b5890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c77fe640 1 -- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 msgr2=0x7f11c80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f1dd5561640 1 -- 172.21.15.148:0/2321532292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00b5fe0 con 0x7f1dd015d5f0 2024-03-04T12:12:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c77fe640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 0x7f11c80cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c77fe640 1 -- 172.21.15.148:0/642100258 >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 msgr2=0x7f11c80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f1dd5561640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c490 tx=0x7f1dcc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f182ea20640 1 Processor -- start 2024-03-04T12:12:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c77fe640 1 --2- 172.21.15.148:0/642100258 >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7fdae8a81640 1 Processor -- start 2024-03-04T12:12:19.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c77fe640 1 -- 172.21.15.148:0/642100258 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c806ffb0 con 0x7f11c815aa90 2024-03-04T12:12:19.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f29d8b8b640 1 -- 172.21.15.148:0/1190150886 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4034070 con 0x7f29cc0382a0 2024-03-04T12:12:19.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c6ffd640 1 --2- 172.21.15.148:0/642100258 >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f29d8b8b640 1 -- 172.21.15.148:0/1190150886 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4030610 con 0x7f29cc0382a0 2024-03-04T12:12:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f11c7fff640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 0x7f11c80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f29d8b8b640 1 -- 172.21.15.148:0/1190150886 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4034070 con 0x7f29cc0382a0 2024-03-04T12:12:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 0x7f33440c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440d0a20 0x7f33440c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 0x7f334414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344168a10 con 0x7f33440d0a20 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440ce9e0 con 0x7f33440cef00 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f334a48c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3344168b90 con 0x7f334415b7c0 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f56dab3c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc051c80 con 0x7f56cc00a510 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f56dab3c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc051f60 con 0x7f56cc051710 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f56dab3c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc052240 con 0x7f56cc0e59a0 2024-03-04T12:12:19.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f56da33b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f56cc00a510 0x7f56cc0e0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47398/0 (socket says 172.21.15.148:47398) 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.758+0000 7f56d9b3a640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f56cc0e59a0 0x7f56cc0e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f56da33b640 1 -- 172.21.15.148:0/3325785418 learned_addr learned my addr 172.21.15.148:0/3325785418 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f3348a29640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 0x7f334414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f3348a29640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 0x7f334414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38626/0 (socket says 172.21.15.148:38626) 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f3348a29640 1 -- 172.21.15.148:0/3119999952 learned_addr learned my addr 172.21.15.148:0/3119999952 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f3343fff640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 0x7f33440c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc005ce0 con 0x7f1dd015d5f0 2024-03-04T12:12:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7ff0beffd640 1 -- 172.21.15.148:0/4289962696 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8039070 con 0x7ff0c0161210 2024-03-04T12:12:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7ff0beffd640 1 -- 172.21.15.148:0/4289962696 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b802cc90 con 0x7ff0c0161210 2024-03-04T12:12:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7ff0beffd640 1 -- 172.21.15.148:0/4289962696 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8034660 con 0x7ff0c0161210 2024-03-04T12:12:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc007070 con 0x7f1dd015d5f0 2024-03-04T12:12:19.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f33437fe640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440d0a20 0x7f33440c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc007460 con 0x7f1dd015d5f0 2024-03-04T12:12:19.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd00b62c0 con 0x7f1dd015d5f0 2024-03-04T12:12:19.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd0155050 con 0x7f1dd015d5f0 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7ff0c63e1640 1 -- 172.21.15.148:0/4289962696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c0161210 msgr2=0x7ff0c0162390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/4289962696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c0161210 0x7ff0c0162390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00b6530 tx=0x7ff0b802bee0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7f182ea20640 1 -- start start 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820033ee0 0x7f18200de700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 0x7f1820047c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820037a80 con 0x7f1820033ee0 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820037900 con 0x7f18200df210 2024-03-04T12:12:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f182ea20640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200df5f0 con 0x7f1820053c60 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f3343fff640 1 -- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 msgr2=0x7f334414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f3343fff640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 0x7f334414dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f3343fff640 1 -- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440d0a20 msgr2=0x7f33440c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f3343fff640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440d0a20 0x7f33440c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f3343fff640 1 -- 172.21.15.148:0/3119999952 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440c70f0 con 0x7f33440cef00 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0382a0 msgr2=0x7f29cc058580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/1190150886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0382a0 0x7f29cc058580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402b960 comp rx=0 tx=0).stop 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.759+0000 7fdae8a81640 1 -- start start 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 0x7fdadc0d9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc0df030 0x7fdadc04a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 0x7fdadc0db630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0507c0 con 0x7fdadc0df410 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc050640 con 0x7fdadc050140 2024-03-04T12:12:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7fdae8a81640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0546c0 con 0x7fdadc0df030 2024-03-04T12:12:19.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7ff0c63e1640 1 -- 172.21.15.148:0/4289962696 shutdown_connections 2024-03-04T12:12:19.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/4289962696 >> v2:172.21.15.148:3300/0 conn(0x7ff0c00b5ca0 0x7ff0c00d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/4289962696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c0161210 0x7ff0c0162390 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00b6530 tx=0x7ff0b802bee0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7ff0c63e1640 1 -- 172.21.15.148:0/4289962696 >> 172.21.15.148:0/4289962696 conn(0x7ff0c00d0590 msgr2=0x7ff0c00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae1d9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc0df030 0x7fdadc04a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 0x7fdadc0d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 0x7fdadc0d9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38132/0 (socket says 172.21.15.148:38132) 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae259c640 1 -- 172.21.15.148:0/1619276805 learned_addr learned my addr 172.21.15.148:0/1619276805 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f3343fff640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 0x7f33440c9d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f333400c3c0 tx=0x7f333400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f182cfbd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 0x7fdadc0db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f1827fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820033ee0 0x7f18200de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f182cfbd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38142/0 (socket says 172.21.15.148:38142) 2024-03-04T12:12:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f11c77fe640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c806ef30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11c804f710 tx=0x7f11c002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33912100abd783d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33340074e0 con 0x7f33440cef00 2024-03-04T12:12:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f182cfbd640 1 -- 172.21.15.148:0/3681952827 learned_addr learned my addr 172.21.15.148:0/3681952827 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f11c67fc640 1 -- 172.21.15.148:0/642100258 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0039070 con 0x7f11c815aa90 2024-03-04T12:12:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f11c67fc640 1 -- 172.21.15.148:0/642100258 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c002cbd0 con 0x7f11c815aa90 2024-03-04T12:12:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f334a48c640 1 -- 172.21.15.148:0/3119999952 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3344155ae0 con 0x7f33440cef00 2024-03-04T12:12:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f11c67fc640 1 -- 172.21.15.148:0/642100258 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0034440 con 0x7f11c815aa90 2024-03-04T12:12:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f18277fe640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 0x7f1820047c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 -- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc0df030 msgr2=0x7fdadc04a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc0df030 0x7fdadc04a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 -- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 msgr2=0x7fdadc0d9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 0x7fdadc0d9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334007680 con 0x7f33440cef00 2024-03-04T12:12:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f334a48c640 1 -- 172.21.15.148:0/3119999952 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33441560a0 con 0x7f33440cef00 2024-03-04T12:12:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7fdae2d9d640 1 -- 172.21.15.148:0/1619276805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc04ab70 con 0x7fdadc0df410 2024-03-04T12:12:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.760+0000 7f56d8b38640 1 -- 172.21.15.148:0/3325785418 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498784572 0 0) 0x7f56cc051f60 con 0x7f56cc051710 2024-03-04T12:12:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f56d9b3a640 1 -- 172.21.15.148:0/3325785418 >> v1:172.21.15.191:6789/0 conn(0x7f56cc051710 legacy=0x7f56cc009e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f56d9b3a640 1 -- 172.21.15.148:0/3325785418 >> v1:172.21.15.148:6789/0 conn(0x7f56cc00a510 legacy=0x7f56cc0e0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 -- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 msgr2=0x7f1820047c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 0x7f1820047c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 -- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820033ee0 msgr2=0x7f18200de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820033ee0 0x7f18200de700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 -- 172.21.15.148:0/3681952827 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820049590 con 0x7f18200df210 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f18277fe640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 0x7f1820047c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f56d9b3a640 1 -- 172.21.15.148:0/3325785418 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc052520 con 0x7f56cc0e59a0 2024-03-04T12:12:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334004a00 con 0x7f33440cef00 2024-03-04T12:12:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f56d9b3a640 1 --2- 172.21.15.148:0/3325785418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0e59a0 0x7f56cc0e4d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56d404f5b0 tx=0x7f56d4068aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1087a42630c38b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182cfbd640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1828064a90 tx=0x7f1828068800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.761+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8005290 con 0x7f1dd015d5f0 2024-03-04T12:12:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7fdae2d9d640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 0x7fdadc0db630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdae406a710 tx=0x7fdae4069fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40bc680 con 0x7fdadc0df410 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc0d7fa0 con 0x7fdadc0df410 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc0d8560 con 0x7fdadc0df410 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7ff0c63e1640 1 -- 172.21.15.148:0/4289962696 shutdown_connections 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7ff0c63e1640 1 -- 172.21.15.148:0/4289962696 wait complete. 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182806ce60 con 0x7f18200df210 2024-03-04T12:12:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18200e22e0 con 0x7f18200df210 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820048ed0 con 0x7f18200df210 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182808e030 con 0x7f18200df210 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828061990 con 0x7f18200df210 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f11cdb9e640 1 -- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 msgr2=0x7f11c806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c806ef30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11c804f710 tx=0x7f11c002be20 comp rx=0 tx=0).stop 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 shutdown_connections 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/1190150886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0382a0 0x7f29cc058580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 >> 172.21.15.148:0/1190150886 conn(0x7f29cc059130 msgr2=0x7f29cc059510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f56d8b38640 1 -- 172.21.15.148:0/3325785418 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40b2070 con 0x7f56cc0e59a0 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f56d8b38640 1 -- 172.21.15.148:0/3325785418 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d40ab830 con 0x7f56cc0e59a0 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.762+0000 7f56d8b38640 1 -- 172.21.15.148:0/3325785418 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d406ce60 con 0x7f56cc0e59a0 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f56dab3c640 1 -- 172.21.15.148:0/3325785418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0e59a0 msgr2=0x7f56cc0e4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f56dab3c640 1 --2- 172.21.15.148:0/3325785418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0e59a0 0x7f56cc0e4d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56d404f5b0 tx=0x7f56d4068aa0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f56dab3c640 1 -- 172.21.15.148:0/3325785418 shutdown_connections 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f56dab3c640 1 --2- 172.21.15.148:0/3325785418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0e59a0 0x7f56cc0e4d30 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56d404f5b0 tx=0x7f56d4068aa0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f56dab3c640 1 -- 172.21.15.148:0/3325785418 >> 172.21.15.148:0/3325785418 conn(0x7f56cc0537a0 msgr2=0x7f56cc053bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3334004ba0 con 0x7f33440cef00 2024-03-04T12:12:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f334a48c640 1 -- 172.21.15.148:0/3119999952 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324005290 con 0x7f33440cef00 2024-03-04T12:12:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 shutdown_connections 2024-03-04T12:12:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.763+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40c1070 con 0x7fdadc0df410 2024-03-04T12:12:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40bca20 con 0x7fdadc0df410 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1808005290 con 0x7f18200df210 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 -- 172.21.15.148:0/642100258 shutdown_connections 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c80cf110 0x7f11c80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/642100258 >> v2:172.21.15.148:3300/0 conn(0x7f11c806f470 0x7f11c80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/642100258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c806ef30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 -- 172.21.15.148:0/642100258 >> 172.21.15.148:0/642100258 conn(0x7f11c80d0780 msgr2=0x7f11c80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f3340ff9640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 0x7f333006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 -- 172.21.15.148:0/642100258 shutdown_connections 2024-03-04T12:12:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f56dab3c640 1 -- 172.21.15.148:0/3325785418 shutdown_connections 2024-03-04T12:12:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f56dab3c640 1 -- 172.21.15.148:0/3325785418 wait complete. 2024-03-04T12:12:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f11cdb9e640 1 -- 172.21.15.148:0/642100258 wait complete. 2024-03-04T12:12:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.764+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f33340898c0 con 0x7f33440cef00 2024-03-04T12:12:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7f29dbdf1640 1 -- 172.21.15.148:0/1190150886 wait complete. 2024-03-04T12:12:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1828061b30 con 0x7f18200df210 2024-03-04T12:12:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7ff0c63e1640 1 Processor -- start 2024-03-04T12:12:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc007ff0 con 0x7fdadc0df410 2024-03-04T12:12:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7ff0c63e1640 1 -- start start 2024-03-04T12:12:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.765+0000 7f1824ff9640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 0x7f180c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1828104950 con 0x7f18200df210 2024-03-04T12:12:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1827fff640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 0x7f180c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1dcc007600 con 0x7f1dd015d5f0 2024-03-04T12:12:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1827fff640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 0x7f180c06da60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f181c006fd0 tx=0x7f181c0090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f56dab3c640 1 Processor -- start 2024-03-04T12:12:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1dc67fc640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 0x7f1db406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1dcc08a0b0 con 0x7f1dd015d5f0 2024-03-04T12:12:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f29dbdf1640 1 Processor -- start 2024-03-04T12:12:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.766+0000 7f56dab3c640 1 -- start start 2024-03-04T12:12:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f33437fe640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 0x7f333006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f11cdb9e640 1 Processor -- start 2024-03-04T12:12:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0c63e1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 0x7ff0c00aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0c63e1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 0x7ff0c00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f1dd5d62640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 0x7f1db406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f29dbdf1640 1 -- start start 2024-03-04T12:12:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f11cdb9e640 1 -- start start 2024-03-04T12:12:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0bffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 0x7ff0c00aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0bffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 0x7ff0c00aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38148/0 (socket says 172.21.15.148:38148) 2024-03-04T12:12:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0bffff640 1 -- 172.21.15.148:0/1183005407 learned_addr learned my addr 172.21.15.148:0/1183005407 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 0x7ff0c00ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f33437fe640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 0x7f333006da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3344161b50 tx=0x7f3338009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0bf7fe640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 0x7ff0c00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7f1dd5d62640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 0x7f1db406da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8036000 tx=0x7f1dc8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.767+0000 7ff0c497e640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 0x7ff0c00ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00cf1d0 con 0x7ff0c0151f40 2024-03-04T12:12:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00ceef0 con 0x7ff0c00b2f30 2024-03-04T12:12:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c01615f0 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29cc058ac0 0x7f29cc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0e3390 0x7f29cc051250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc058060 con 0x7f29cc0e3390 2024-03-04T12:12:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0540a0 con 0x7f29cc0382a0 2024-03-04T12:12:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56dab3c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29dbdf1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc054220 con 0x7f29cc058ac0 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38156/0 (socket says 172.21.15.148:38156) 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 -- 172.21.15.148:0/4029294283 learned_addr learned my addr 172.21.15.148:0/4029294283 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d938c640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29cc058ac0 0x7f29cc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56dab3c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 0x7f56cc02d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56d9b3a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f11cdb9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c806f470 0x7f11c80c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f11cdb9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 -- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29cc058ac0 msgr2=0x7f29cc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56d9b3a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38680/0 (socket says 172.21.15.148:38680) 2024-03-04T12:12:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56d9b3a640 1 -- 172.21.15.148:0/2212329077 learned_addr learned my addr 172.21.15.148:0/2212329077 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29cc058ac0 0x7f29cc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 -- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0e3390 msgr2=0x7f29cc051250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0e3390 0x7f29cc051250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0d7370 0x7f56cc0558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0bf7fe640 1 -- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 msgr2=0x7ff0c00aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0bf7fe640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 0x7ff0c00aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0bf7fe640 1 -- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 msgr2=0x7ff0c00ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f29d9b8d640 1 -- 172.21.15.148:0/4029294283 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc051a80 con 0x7f29cc0382a0 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0e40e0 con 0x7f56cc051710 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0e3f60 con 0x7f56cc0d7370 2024-03-04T12:12:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0e4260 con 0x7f56cc00a510 2024-03-04T12:12:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0bf7fe640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 0x7ff0c00ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7ff0bf7fe640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00ce0e0 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f29d9b8d640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29d000c480 tx=0x7f29d000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0c497e640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 0x7ff0c00ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40064/0 (socket says 172.21.15.148:40064) 2024-03-04T12:12:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 -- 172.21.15.148:0/2442891042 learned_addr learned my addr 172.21.15.148:0/2442891042 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56da33b640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0d7370 0x7f56cc0558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c815d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.768+0000 7f56d9339640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 0x7f56cc02d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80afda0 con 0x7f11c80cf110 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c815ae70 con 0x7f11c815aa90 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80cd810 con 0x7f11c806f470 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c7fff640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 -- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0d7370 msgr2=0x7f56cc0558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0d7370 0x7f56cc0558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 -- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 msgr2=0x7f56cc02d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0bf7fe640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 0x7ff0c00cdba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000f60 tx=0x7ff0b000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 0x7f56cc02d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0d47d0 con 0x7f56cc00a510 2024-03-04T12:12:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0017070 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0005ce0 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0007310 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c77fe640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c806f470 0x7f11c80c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c00cabe0 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9339640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 0x7f56cc02d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f56d9b3a640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56d402d440 tx=0x7f56d4062b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 -- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c806f470 msgr2=0x7f11c80c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c806f470 0x7f11c80c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 -- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 msgr2=0x7f11c815d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c815d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.769+0000 7f11c6ffd640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c815cbc0 con 0x7f11c80cf110 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40ba020 con 0x7f56cc00a510 2024-03-04T12:12:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d406be50 con 0x7f56cc00a510 2024-03-04T12:12:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f11c6ffd640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ebb0 tx=0x7f11bc00c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40b2040 con 0x7f56cc00a510 2024-03-04T12:12:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc02dd10 con 0x7f56cc00a510 2024-03-04T12:12:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc0d65b0 con 0x7f56cc00a510 2024-03-04T12:12:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00ab770 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.770+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3334057d00 con 0x7f33440cef00 2024-03-04T12:12:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.771+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0057d0 con 0x7f56cc00a510 2024-03-04T12:12:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.771+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c0074f60 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.771+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1dcc058c60 con 0x7f1dd015d5f0 2024-03-04T12:12:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff0b001a070 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f56d40bd020 con 0x7f56cc00a510 2024-03-04T12:12:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7ff0bcff9640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 0x7ff0ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc004030 con 0x7f11c80cf110 2024-03-04T12:12:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc0041d0 con 0x7f11c80cf110 2024-03-04T12:12:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00c980 con 0x7f11c80cf110 2024-03-04T12:12:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff0b0089700 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c815c600 con 0x7f11c80cf110 2024-03-04T12:12:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.772+0000 7f56ca7fc640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 0x7f56b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f56d4126ff0 con 0x7f56cc00a510 2024-03-04T12:12:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80b3a00 con 0x7f11c80cf110 2024-03-04T12:12:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdae40bcbc0 con 0x7fdadc0df410 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7fdad2ffd640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 0x7fdac406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdae412a1f0 con 0x7fdadc0df410 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.773+0000 7fdae259c640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 0x7fdac406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.774+0000 7f56d9339640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 0x7f56b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.774+0000 7fdae259c640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 0x7fdac406da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009a30 tx=0x7fdad4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.774+0000 7ff0bffff640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 0x7ff0ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.774+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8074f60 con 0x7f11c80cf110 2024-03-04T12:12:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7ff0bffff640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 0x7ff0ac06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009bd0 tx=0x7ff0b8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f56d9339640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 0x7f56b806d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56cc050df0 tx=0x7f56c4007db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18280d3500 con 0x7f18200df210 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0017070 con 0x7f29cc0382a0 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0005ce0 con 0x7f29cc0382a0 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00072f0 con 0x7f29cc0382a0 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc051d60 con 0x7f29cc0382a0 2024-03-04T12:12:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.775+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc052320 con 0x7f29cc0382a0 2024-03-04T12:12:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.776+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc0e1230 con 0x7f29cc0382a0 2024-03-04T12:12:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.776+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56d40c4030 con 0x7f56cc00a510 2024-03-04T12:12:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.777+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f29d001a070 con 0x7f29cc0382a0 2024-03-04T12:12:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.777+0000 7f29ca7fc640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 0x7f29b806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.777+0000 7f29d938c640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 0x7f29b806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.777+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f29d00896b0 con 0x7f29cc0382a0 2024-03-04T12:12:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.778+0000 7f29d938c640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 0x7f29b806da10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29c4036000 tx=0x7f29c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.781+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0b0057c80 con 0x7ff0c00b5ca0 2024-03-04T12:12:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.781+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdadc007ff0 con 0x7fdadc0df410 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.784+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f11bc00cb20 con 0x7f11c80cf110 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.784+0000 7f11affff640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 0x7f11b406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.785+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f11bc08ac40 con 0x7f11c80cf110 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.785+0000 7f11c77fe640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 0x7f11b406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.786+0000 7f11c77fe640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 0x7f11b406da60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11c0036000 tx=0x7f11c0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.786+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29d0057d10 con 0x7f29cc0382a0 2024-03-04T12:12:19.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:19.792+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11bc017020 con 0x7f11c80cf110 2024-03-04T12:12:20.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.179+0000 7f334a48c640 1 -- 172.21.15.148:0/3119999952 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3324005580 con 0x7f33440cef00 2024-03-04T12:12:20.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.179+0000 7f3340ff9640 1 -- 172.21.15.148:0/3119999952 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f333405bda0 con 0x7f33440cef00 2024-03-04T12:12:20.182 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.182+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1db8005580 con 0x7f1dd015d5f0 2024-03-04T12:12:20.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.182+0000 7f1dc67fc640 1 -- 172.21.15.148:0/2321532292 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1dcc058560 con 0x7f1dd015d5f0 2024-03-04T12:12:20.185 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 msgr2=0x7f333006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 0x7f333006da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3344161b50 tx=0x7f3338009040 comp rx=0 tx=0).stop 2024-03-04T12:12:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 msgr2=0x7f33440c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 0x7f33440c9d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f333400c3c0 tx=0x7f333400c880 comp rx=0 tx=0).stop 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 shutdown_connections 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f333006b5a0 0x7f333006da60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f334415b7c0 0x7f334414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33440d0a20 0x7f33440c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 --2- 172.21.15.148:0/3119999952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33440cef00 0x7f33440c9d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.183+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 >> 172.21.15.148:0/3119999952 conn(0x7f3344072400 msgr2=0x7f334415cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.184+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 shutdown_connections 2024-03-04T12:12:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.184+0000 7f332a7fc640 1 -- 172.21.15.148:0/3119999952 wait complete. 2024-03-04T12:12:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.184+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f29cc051790 con 0x7f29cc0382a0 2024-03-04T12:12:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.184+0000 7f29ca7fc640 1 -- 172.21.15.148:0/4029294283 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29d005bdb0 con 0x7f29cc0382a0 2024-03-04T12:12:20.188 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.186+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 msgr2=0x7f1db406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.186+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 0x7f1db406da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8036000 tx=0x7f1dc8030000 comp rx=0 tx=0).stop 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.186+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 msgr2=0x7f1dd00c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.186+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c490 tx=0x7f1dcc00c950 comp rx=0 tx=0).stop 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 shutdown_connections 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1db406b5a0 0x7f1db406da60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd015da60 0x7f1dd00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd015d5f0 0x7f1dd00c9170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 --2- 172.21.15.148:0/2321532292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd015ce00 0x7f1dd01572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 >> 172.21.15.148:0/2321532292 conn(0x7f1dd00b3180 msgr2=0x7f1dd00aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 shutdown_connections 2024-03-04T12:12:20.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.187+0000 7f1dd7fc6640 1 -- 172.21.15.148:0/2321532292 wait complete. 2024-03-04T12:12:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.188+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 msgr2=0x7f29b806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.188+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 0x7f29b806da10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29c4036000 tx=0x7f29c4008030 comp rx=0 tx=0).stop 2024-03-04T12:12:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 msgr2=0x7f29cc0e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29d000c480 tx=0x7f29d000c940 comp rx=0 tx=0).stop 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 shutdown_connections 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29b806b550 0x7f29b806da10 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29c4036000 tx=0x7f29c4008030 comp rx=0 tx=0).stop 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29cc0e3390 0x7f29cc051250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29cc058ac0 0x7f29cc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 --2- 172.21.15.148:0/4029294283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29cc0382a0 0x7f29cc0e5900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 >> 172.21.15.148:0/4029294283 conn(0x7f29cc059130 msgr2=0x7f29cc03dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 shutdown_connections 2024-03-04T12:12:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.189+0000 7f29dbdf1640 1 -- 172.21.15.148:0/4029294283 wait complete. 2024-03-04T12:12:20.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.191+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1808005580 con 0x7f18200df210 2024-03-04T12:12:20.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.191+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f56cc0512e0 con 0x7f56cc00a510 2024-03-04T12:12:20.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.191+0000 7f1824ff9640 1 -- 172.21.15.148:0/3681952827 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18280d2e00 con 0x7f18200df210 2024-03-04T12:12:20.194 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.194+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdadc0053c0 con 0x7fdadc0df410 2024-03-04T12:12:20.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.194+0000 7f56ca7fc640 1 -- 172.21.15.148:0/2212329077 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56d40f5510 con 0x7f56cc00a510 2024-03-04T12:12:20.197 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1619276805 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdae40f86c0 con 0x7fdadc0df410 2024-03-04T12:12:20.198 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:20.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 msgr2=0x7f180c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 0x7f180c06da60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f181c006fd0 tx=0x7f181c0090e0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff0c00d1890 con 0x7ff0c00b5ca0 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 msgr2=0x7f182017ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.195+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1828064a90 tx=0x7f1828068800 comp rx=0 tx=0).stop 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 shutdown_connections 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f180c06b5a0 0x7f180c06da60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18200df210 0x7f182017ca40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1820053c60 0x7f1820047c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 --2- 172.21.15.148:0/3681952827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1820033ee0 0x7f18200de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 >> 172.21.15.148:0/3681952827 conn(0x7f1820009fa0 msgr2=0x7f18200e2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 shutdown_connections 2024-03-04T12:12:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7f182ea20640 1 -- 172.21.15.148:0/3681952827 wait complete. 2024-03-04T12:12:20.201 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-04T12:12:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.196+0000 7ff0bcff9640 1 -- 172.21.15.148:0/1183005407 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0b005bd20 con 0x7ff0c00b5ca0 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 msgr2=0x7f56b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 0x7f56b806d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56cc050df0 tx=0x7f56c4007db0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 msgr2=0x7f56cc0d4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56d402d440 tx=0x7f56d4062b00 comp rx=0 tx=0).stop 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 shutdown_connections 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f56b806b500 0x7f56b806d9c0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56cc050df0 tx=0x7f56c4007db0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56cc0d7370 0x7f56cc0558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc051710 0x7f56cc02d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 --2- 172.21.15.148:0/2212329077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56cc00a510 0x7f56cc0d4290 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.198+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 >> 172.21.15.148:0/2212329077 conn(0x7f56cc0537a0 msgr2=0x7f56cc0e21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 msgr2=0x7fdac406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 0x7fdac406da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009a30 tx=0x7fdad4009190 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 msgr2=0x7fdadc0db630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 0x7fdadc0db630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdae406a710 tx=0x7fdae4069fc0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 shutdown_connections 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdac406b5a0 0x7fdac406da60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0df410 0x7fdadc0db630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdadc0df030 0x7fdadc04a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 --2- 172.21.15.148:0/1619276805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdadc050140 0x7fdadc0d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 >> 172.21.15.148:0/1619276805 conn(0x7fdadc009fe0 msgr2=0x7fdadc052ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 shutdown_connections 2024-03-04T12:12:20.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7fdae8a81640 1 -- 172.21.15.148:0/1619276805 wait complete. 2024-03-04T12:12:20.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 shutdown_connections 2024-03-04T12:12:20.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.199+0000 7f56dab3c640 1 -- 172.21.15.148:0/2212329077 wait complete. 2024-03-04T12:12:20.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.200+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f11c80ce3c0 con 0x7f11c80cf110 2024-03-04T12:12:20.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7f11affff640 1 -- 172.21.15.148:0/2442891042 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11bc0597f0 con 0x7f11c80cf110 2024-03-04T12:12:20.206 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-04T12:12:20.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.200+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 msgr2=0x7ff0ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 0x7ff0ac06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009bd0 tx=0x7ff0b8030000 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 msgr2=0x7ff0c00cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 0x7ff0c00cdba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000f60 tx=0x7ff0b000c800 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 shutdown_connections 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff0ac06b4d0 0x7ff0ac06d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c0151f40 0x7ff0c00ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c00b5ca0 0x7ff0c00cdba0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000f60 tx=0x7ff0b000c800 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 --2- 172.21.15.148:0/1183005407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00b2f30 0x7ff0c00aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.201+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 >> 172.21.15.148:0/1183005407 conn(0x7ff0c00d0590 msgr2=0x7ff0c00d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.202+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 shutdown_connections 2024-03-04T12:12:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.202+0000 7ff0c63e1640 1 -- 172.21.15.148:0/1183005407 wait complete. 2024-03-04T12:12:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 msgr2=0x7f11b406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 0x7f11b406da60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11c0036000 tx=0x7f11c0030000 comp rx=0 tx=0).stop 2024-03-04T12:12:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 msgr2=0x7f11c814c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ebb0 tx=0x7f11bc00c150 comp rx=0 tx=0).stop 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 shutdown_connections 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f11b406b5a0 0x7f11b406da60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11c815aa90 0x7f11c815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11c80cf110 0x7f11c814c6d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 --2- 172.21.15.148:0/2442891042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11c806f470 0x7f11c80c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:20.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.205+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 >> 172.21.15.148:0/2442891042 conn(0x7f11c80d0780 msgr2=0x7f11c80ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:20.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.206+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 shutdown_connections 2024-03-04T12:12:20.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:20.206+0000 7f11cdb9e640 1 -- 172.21.15.148:0/2442891042 wait complete. 2024-03-04T12:12:20.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-03-04T12:12:20.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-03-04T12:12:20.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-03-04T12:12:20.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-03-04T12:12:20.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-03-04T12:12:20.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-03-04T12:12:20.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-03-04T12:12:20.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-03-04T12:12:21.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T12:12:21.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T12:12:21.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T12:12:21.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T12:12:21.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T12:12:21.226 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T12:12:21.227 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T12:12:21.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T12:12:21.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.374+0000 7f5b2875b640 1 Processor -- start 2024-03-04T12:12:21.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.374+0000 7f5b2875b640 1 -- start start 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b2875b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b200b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b2875b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200cf7d0 con 0x7f5b200aefb0 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b2875b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200cfab0 con 0x7f5b200cf340 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b2875b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2016acb0 con 0x7f5b20071a60 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b264f7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5b200aefb0 0x7f5b20071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47408/0 (socket says 172.21.15.148:47408) 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b26cf8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b200b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b25cf6640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f5b200cf340 0x7f5b200b2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58688/0 (socket says 172.21.15.148:58688) 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.376+0000 7f5b264f7640 1 -- 172.21.15.148:0/1276522692 learned_addr learned my addr 172.21.15.148:0/1276522692 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.377+0000 7f5b254f5640 1 -- 172.21.15.148:0/1276522692 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217990097 0 0) 0x7f5b200cfab0 con 0x7f5b200cf340 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.377+0000 7f5b26cf8640 1 -- 172.21.15.148:0/1276522692 >> v1:172.21.15.191:6789/0 conn(0x7f5b200cf340 legacy=0x7f5b200b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.377+0000 7f5b26cf8640 1 -- 172.21.15.148:0/1276522692 >> v1:172.21.15.148:6789/0 conn(0x7f5b200aefb0 legacy=0x7f5b20071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.377+0000 7f5b26cf8640 1 -- 172.21.15.148:0/1276522692 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200cfd90 con 0x7f5b20071a60 2024-03-04T12:12:21.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.378+0000 7f0b58a59640 1 Processor -- start 2024-03-04T12:12:21.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.378+0000 7f0b58a59640 1 -- start start 2024-03-04T12:12:21.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.378+0000 7f5b26cf8640 1 --2- 172.21.15.148:0/1276522692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b200b31f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009dc0 tx=0x7f5b1802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c37bfb6cf38c08e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f0b58a59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b5415de00 0x7f0b540b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f0b58a59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b540b9380 0x7f0b540b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f0b58a59640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5415e1e0 con 0x7f0b540b70d0 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f0b58a59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540d2820 con 0x7f0b540b9380 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f0b58a59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d1c10 con 0x7f0b5415de00 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f0b52d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0b540b70d0 0x7f0b540b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47418/0 (socket says 172.21.15.148:47418) 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f0b52d9d640 1 -- 172.21.15.148:0/1832329401 learned_addr learned my addr 172.21.15.148:0/1832329401 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f0b5259c640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b5415de00 0x7f0b540b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f5b254f5640 1 -- 172.21.15.148:0/1276522692 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18039070 con 0x7f5b20071a60 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f5b254f5640 1 -- 172.21.15.148:0/1276522692 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1802cc50 con 0x7f5b20071a60 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.379+0000 7f5b254f5640 1 -- 172.21.15.148:0/1276522692 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18034440 con 0x7f5b20071a60 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 -- 172.21.15.148:0/1276522692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 msgr2=0x7f5b200b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 --2- 172.21.15.148:0/1276522692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b200b31f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009dc0 tx=0x7f5b1802bea0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 -- 172.21.15.148:0/1276522692 shutdown_connections 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 --2- 172.21.15.148:0/1276522692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b200b31f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 -- 172.21.15.148:0/1276522692 >> 172.21.15.148:0/1276522692 conn(0x7f5b2006acd0 msgr2=0x7f5b2006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 -- 172.21.15.148:0/1276522692 shutdown_connections 2024-03-04T12:12:21.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f5b2875b640 1 -- 172.21.15.148:0/1276522692 wait complete. 2024-03-04T12:12:21.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405336363 0 0) 0x7f0b5415e1e0 con 0x7f0b540b70d0 2024-03-04T12:12:21.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.380+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b3c003520 con 0x7f0b540b70d0 2024-03-04T12:12:21.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092708186 0 0) 0x7f0b3c003520 con 0x7f0b540b70d0 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b5415de00 msgr2=0x7f0b540b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b5415de00 0x7f0b540b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 >> v2:172.21.15.148:3300/0 conn(0x7f0b540b9380 msgr2=0x7f0b540b6b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 --2- 172.21.15.148:0/1832329401 >> v2:172.21.15.148:3300/0 conn(0x7f0b540b9380 0x7f0b540b6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540d1ef0 con 0x7f0b540b70d0 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f0b4c0019f0 con 0x7f0b540b70d0 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 >> v1:172.21.15.148:6789/0 conn(0x7f0b540b70d0 legacy=0x7f0b540b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b3c0039b0 0x7f0b3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b3c004800 0x7f0b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0051c0 0x7f0b3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d2820 con 0x7f0b3c0039b0 2024-03-04T12:12:21.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d1c10 con 0x7f0b3c004800 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c00c060 con 0x7f0b3c0051c0 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.381+0000 7f0b52d9d640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0051c0 0x7f0b3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.382+0000 7f0b5259c640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b3c0039b0 0x7f0b3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.382+0000 7f0b5259c640 1 -- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0051c0 msgr2=0x7f0b3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.382+0000 7f0b5259c640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0051c0 0x7f0b3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.382+0000 7f0b5259c640 1 -- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b3c004800 msgr2=0x7f0b3c004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.382+0000 7f0b5259c640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b3c004800 0x7f0b3c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.383+0000 7f0b5259c640 1 -- 172.21.15.148:0/1832329401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c003520 con 0x7f0b3c0039b0 2024-03-04T12:12:21.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.384+0000 7f0b5259c640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b3c0039b0 0x7f0b3c003dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406d3e0 tx=0x7f0b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bebf177aec89d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44039070 con 0x7f0b3c0039b0 2024-03-04T12:12:21.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f0b5159a640 1 -- 172.21.15.148:0/1832329401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4402c7d0 con 0x7f0b3c0039b0 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.383+0000 7f5b2875b640 1 Processor -- start 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.383+0000 7f5b2875b640 1 -- start start 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.384+0000 7f5b2875b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b2016da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.384+0000 7f5b2875b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 0x7f5b202155c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.384+0000 7f5b2875b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.384+0000 7f5b26cf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b26cf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40082/0 (socket says 172.21.15.148:40082) 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b26cf8640 1 -- 172.21.15.148:0/3578293213 learned_addr learned my addr 172.21.15.148:0/3578293213 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20072090 con 0x7f5b2015c4a0 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20071f10 con 0x7f5b200aefb0 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2016e330 con 0x7f5b20071a60 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f0b58a59640 1 -- 172.21.15.148:0/1832329401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b3c0039b0 msgr2=0x7f0b3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f0b58a59640 1 --2- 172.21.15.148:0/1832329401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b3c0039b0 0x7f0b3c003dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406d3e0 tx=0x7f0b3c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f0b52d9d640 1 -- 172.21.15.148:0/1832329401 reap_dead start 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b25cf6640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 0x7f5b202155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b25cf6640 1 -- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 msgr2=0x7f5b2016da40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b25cf6640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b2016da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.385+0000 7f5b25cf6640 1 -- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 msgr2=0x7f5b20164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f5b25cf6640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f5b25cf6640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2015ca60 con 0x7f5b200aefb0 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f5b26cf8640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.386+0000 7f5b25cf6640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 0x7f5b202155c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00c3e0 tx=0x7f5b1c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.387+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c0074e0 con 0x7f5b200aefb0 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.387+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2015cd40 con 0x7f5b200aefb0 2024-03-04T12:12:21.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.387+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2015d300 con 0x7f5b200aefb0 2024-03-04T12:12:21.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.388+0000 7f0b58a59640 1 -- 172.21.15.148:0/1832329401 shutdown_connections 2024-03-04T12:12:21.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.388+0000 7f0b58a59640 1 -- 172.21.15.148:0/1832329401 >> 172.21.15.148:0/1832329401 conn(0x7f0b540d0470 msgr2=0x7f0b540d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.388+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20074f60 con 0x7f5b200aefb0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.388+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c0047e0 con 0x7f5b200aefb0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.388+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c004c10 con 0x7f5b200aefb0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.389+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5b1c011070 con 0x7f5b200aefb0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.389+0000 7f5b16ffd640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 0x7f5b0406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.390+0000 7f5b264f7640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 0x7f5b0406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.390+0000 7f5b264f7640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 0x7f5b0406da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b100015d0 tx=0x7f5b10007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.390+0000 7f0b58a59640 1 -- 172.21.15.148:0/1832329401 shutdown_connections 2024-03-04T12:12:21.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.391+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5b1c08a0c0 con 0x7f5b200aefb0 2024-03-04T12:12:21.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.391+0000 7f0b58a59640 1 -- 172.21.15.148:0/1832329401 wait complete. 2024-03-04T12:12:21.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.393+0000 7f0b58a59640 1 Processor -- start 2024-03-04T12:12:21.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.393+0000 7f0b58a59640 1 -- start start 2024-03-04T12:12:21.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.394+0000 7f0b58a59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.394+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b1c00e280 con 0x7f5b200aefb0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.394+0000 7f0b58a59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 0x7f0b540b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.394+0000 7f0b58a59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b540b43f0 0x7f0b541f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b58a59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b97e0 con 0x7f0b5414a760 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:38706/0 (socket says 172.21.15.148:38706) 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b5259c640 1 -- 172.21.15.148:0/2358750287 learned_addr learned my addr 172.21.15.148:0/2358750287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b52d9d640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b540b43f0 0x7f0b541f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b58a59640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d24b0 con 0x7f0b540b43f0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b58a59640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b5b80 con 0x7f0b3c0039b0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.395+0000 7f0b51d9b640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 0x7f0b540b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 -- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b540b43f0 msgr2=0x7f0b541f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b540b43f0 0x7f0b541f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 -- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 msgr2=0x7f0b540b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 0x7f0b540b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b541f7990 con 0x7f0b3c0039b0 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.396+0000 7f0b5259c640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b4402c440 tx=0x7f0b44004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.397+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44039070 con 0x7f0b3c0039b0 2024-03-04T12:12:21.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.397+0000 7f0b51d9b640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 0x7f0b540b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.397+0000 7f0b58a59640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54149c80 con 0x7f0b3c0039b0 2024-03-04T12:12:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.397+0000 7f0b58a59640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5414a240 con 0x7f0b3c0039b0 2024-03-04T12:12:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.399+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4403a040 con 0x7f0b3c0039b0 2024-03-04T12:12:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.399+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440408c0 con 0x7f0b3c0039b0 2024-03-04T12:12:21.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.400+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0b4402c7d0 con 0x7f0b3c0039b0 2024-03-04T12:12:21.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.400+0000 7f0b42ffd640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 0x7f0b3406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.401+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b440ac140 con 0x7f0b3c0039b0 2024-03-04T12:12:21.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.401+0000 7f0b51d9b640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 0x7f0b3406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.402+0000 7f0b51d9b640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 0x7f0b3406da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b540b2cd0 tx=0x7f0b48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.403+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b540b4800 con 0x7f0b3c0039b0 2024-03-04T12:12:21.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.409+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b540b4800 con 0x7f0b3c0039b0 2024-03-04T12:12:21.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.427+0000 7f468b59e640 1 Processor -- start 2024-03-04T12:12:21.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.427+0000 7f468b59e640 1 -- start start 2024-03-04T12:12:21.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f468b59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46840069d0 0x7f4684006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f468b59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4684053710 con 0x7f46840eacf0 2024-03-04T12:12:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f468b59e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46840539f0 con 0x7f4684005e80 2024-03-04T12:12:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f468b59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4684054320 con 0x7f46840069d0 2024-03-04T12:12:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f4689d9b640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4684005e80 0x7f46840062c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58694/0 (socket says 172.21.15.148:58694) 2024-03-04T12:12:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f4689d9b640 1 -- 172.21.15.148:0/458977885 learned_addr learned my addr 172.21.15.148:0/458977885 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.429+0000 7f468ad9d640 1 --2- 172.21.15.148:0/458977885 >> v2:172.21.15.148:3300/0 conn(0x7f46840069d0 0x7f4684006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.430+0000 7f468959a640 1 -- 172.21.15.148:0/458977885 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280187163 0 0) 0x7f46840539f0 con 0x7f4684005e80 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.430+0000 7f468959a640 1 -- 172.21.15.148:0/458977885 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4674003520 con 0x7f4684005e80 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.430+0000 7f468ad9d640 1 -- 172.21.15.148:0/458977885 >> v1:172.21.15.191:6789/0 conn(0x7f4684005e80 legacy=0x7f46840062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.430+0000 7f468ad9d640 1 -- 172.21.15.148:0/458977885 >> v1:172.21.15.148:6789/0 conn(0x7f46840eacf0 legacy=0x7f468404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.430+0000 7f468ad9d640 1 -- 172.21.15.148:0/458977885 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684053cd0 con 0x7f46840069d0 2024-03-04T12:12:21.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7f47d359e640 1 Processor -- start 2024-03-04T12:12:21.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7f468ad9d640 1 --2- 172.21.15.148:0/458977885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f4684006e40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46800097a0 tx=0x7f468002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53412cc06afa09f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7f47d359e640 1 -- start start 2024-03-04T12:12:21.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7fbd8a001640 1 Processor -- start 2024-03-04T12:12:21.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7fbd8a001640 1 -- start start 2024-03-04T12:12:21.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.431+0000 7f468959a640 1 -- 172.21.15.148:0/458977885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680035020 con 0x7f46840069d0 2024-03-04T12:12:21.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.432+0000 7f468959a640 1 -- 172.21.15.148:0/458977885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4680031830 con 0x7f46840069d0 2024-03-04T12:12:21.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 -- 172.21.15.148:0/458977885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 msgr2=0x7f4684006e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd8a001640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd840cfdb0 0x7fbd8406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd8a001640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8415f780 con 0x7fbd841447d0 2024-03-04T12:12:21.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd8a001640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8415fa60 con 0x7fbd84144e00 2024-03-04T12:12:21.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd8a001640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd84144bb0 con 0x7fbd840cfdb0 2024-03-04T12:12:21.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 --2- 172.21.15.148:0/458977885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f4684006e40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46800097a0 tx=0x7f468002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d359e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f47cc0e75e0 0x7f47cc03bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f0425fe9640 1 Processor -- start 2024-03-04T12:12:21.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47cc0554f0 0x7f47cc053500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d359e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc057cd0 con 0x7f47cc056db0 2024-03-04T12:12:21.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 -- 172.21.15.148:0/458977885 shutdown_connections 2024-03-04T12:12:21.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 --2- 172.21.15.148:0/458977885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f4684006e40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 -- 172.21.15.148:0/458977885 >> 172.21.15.148:0/458977885 conn(0x7f4684009fa0 msgr2=0x7f468400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d359e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc058db0 con 0x7f47cc0554f0 2024-03-04T12:12:21.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d359e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc006860 con 0x7f47cc0e75e0 2024-03-04T12:12:21.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 -- 172.21.15.148:0/458977885 shutdown_connections 2024-03-04T12:12:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d2d9d640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f47cc056db0 0x7f47cc006cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58704/0 (socket says 172.21.15.148:58704) 2024-03-04T12:12:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d2d9d640 1 -- 172.21.15.148:0/99039390 learned_addr learned my addr 172.21.15.148:0/99039390 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d259c640 1 --2- 172.21.15.148:0/99039390 >> v2:172.21.15.191:3300/0 conn(0x7f47cc0e75e0 0x7f47cc03bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/99039390 >> v2:172.21.15.148:3300/0 conn(0x7f47cc0554f0 0x7f47cc053500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd82ffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd840cfdb0 0x7fbd8406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd82ffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd840cfdb0 0x7fbd8406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38222/0 (socket says 172.21.15.148:38222) 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7fbd82ffd640 1 -- 172.21.15.148:0/809878703 learned_addr learned my addr 172.21.15.148:0/809878703 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7fbd82ffd640 1 -- 172.21.15.148:0/809878703 >> v1:172.21.15.191:6789/0 conn(0x7fbd84144e00 legacy=0x7fbd840cc080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7fbd82ffd640 1 -- 172.21.15.148:0/809878703 >> v1:172.21.15.148:6789/0 conn(0x7fbd841447d0 legacy=0x7fbd840cf6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T12:12:21.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f9718cd1640 1 Processor -- start 2024-03-04T12:12:21.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7ff3f9e27640 1 Processor -- start 2024-03-04T12:12:21.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f9718cd1640 1 -- start start 2024-03-04T12:12:21.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f0425fe9640 1 -- start start 2024-03-04T12:12:21.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f0425fe9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 0x7f0418057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7fbd82ffd640 1 -- 172.21.15.148:0/809878703 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8415fd40 con 0x7fbd840cfdb0 2024-03-04T12:12:21.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f0425fe9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 0x7f0418053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f0425fe9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180582d0 con 0x7f0418054130 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f47d1d9b640 1 -- 172.21.15.148:0/99039390 >> v2:172.21.15.191:3300/0 conn(0x7f47cc0e75e0 msgr2=0x7f47cc03bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/99039390 >> v2:172.21.15.191:3300/0 conn(0x7f47cc0e75e0 0x7f47cc03bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f47d1d9b640 1 -- 172.21.15.148:0/99039390 >> v1:172.21.15.191:6789/0 conn(0x7f47cc056db0 legacy=0x7f47cc006cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7f47d1d9b640 1 -- 172.21.15.148:0/99039390 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc005f30 con 0x7f47cc0554f0 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f0425fe9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0418058570 con 0x7f041803e840 2024-03-04T12:12:21.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f0425fe9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0418058850 con 0x7f04180e30c0 2024-03-04T12:12:21.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/99039390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0554f0 0x7f47cc053500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009af0 tx=0x7f47c402bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21944db798a40349 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7fbd82ffd640 1 --2- 172.21.15.148:0/809878703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd840cfdb0 0x7fbd8406ecb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009870 tx=0x7fbd7802b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd30651f33bfccfb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.434+0000 7ff3f9e27640 1 -- start start 2024-03-04T12:12:21.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f04257e8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0418054130 0x7f04180e6c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47444/0 (socket says 172.21.15.148:47444) 2024-03-04T12:12:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f04257e8640 1 -- 172.21.15.148:0/650439708 learned_addr learned my addr 172.21.15.148:0/650439708 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.433+0000 7f468b59e640 1 -- 172.21.15.148:0/458977885 wait complete. 2024-03-04T12:12:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7f468b59e640 1 Processor -- start 2024-03-04T12:12:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7ff3f9e27640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff3f406e1c0 0x7ff3f4071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.435+0000 7ff3f9e27640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3f415b510 0x7ff3f4071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f9e27640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f406f240 con 0x7ff3f415a700 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f9e27640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f40721d0 con 0x7ff3f415b510 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f9e27640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f406e6c0 con 0x7ff3f406e1c0 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7fbd827fc640 1 -- 172.21.15.148:0/809878703 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7802e070 con 0x7fbd840cfdb0 2024-03-04T12:12:21.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7fbd827fc640 1 -- 172.21.15.148:0/809878703 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78037070 con 0x7fbd840cfdb0 2024-03-04T12:12:21.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f0424fe7640 1 --2- 172.21.15.148:0/650439708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 0x7f0418057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f37fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff3f415a700 0x7ff3f415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:58710/0 (socket says 172.21.15.148:58710) 2024-03-04T12:12:21.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f37fe640 1 -- 172.21.15.148:0/838909059 learned_addr learned my addr 172.21.15.148:0/838909059 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f3fff640 1 --2- 172.21.15.148:0/838909059 >> v2:172.21.15.148:3300/0 conn(0x7ff3f415b510 0x7ff3f4071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9718cd1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9714161df0 0x7f971406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9718cd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f971406b3e0 0x7f97140b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9718cd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97141668c0 con 0x7f97140aec20 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9718cd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97140cf2b0 con 0x7f971406b3e0 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9718cd1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9714166390 con 0x7f9714161df0 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f971259c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9714161df0 0x7f971406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9711d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97140aec20 0x7f971406acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47446/0 (socket says 172.21.15.148:47446) 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f9711d9b640 1 -- 172.21.15.148:0/1857443671 learned_addr learned my addr 172.21.15.148:0/1857443671 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7f468b59e640 1 -- start start 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f27fc640 1 -- 172.21.15.148:0/838909059 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154617216 0 0) 0x7ff3f406f240 con 0x7ff3f415a700 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f27fc640 1 -- 172.21.15.148:0/838909059 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3dc003520 con 0x7ff3f415a700 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f3fff640 1 -- 172.21.15.148:0/838909059 >> v2:172.21.15.191:3300/0 conn(0x7ff3f406e1c0 msgr2=0x7ff3f4071350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f3fff640 1 --2- 172.21.15.148:0/838909059 >> v2:172.21.15.191:3300/0 conn(0x7ff3f406e1c0 0x7ff3f4071350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f3fff640 1 -- 172.21.15.148:0/838909059 >> v1:172.21.15.191:6789/0 conn(0x7ff3f415a700 legacy=0x7ff3f415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7ff3f3fff640 1 -- 172.21.15.148:0/838909059 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f406e9a0 con 0x7ff3f415b510 2024-03-04T12:12:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.436+0000 7fbd827fc640 1 -- 172.21.15.148:0/809878703 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78031a00 con 0x7fbd840cfdb0 2024-03-04T12:12:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f971259c640 1 -- 172.21.15.148:0/1857443671 >> v2:172.21.15.148:3300/0 conn(0x7f971406b3e0 msgr2=0x7f97140b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f971259c640 1 --2- 172.21.15.148:0/1857443671 >> v2:172.21.15.148:3300/0 conn(0x7f971406b3e0 0x7f97140b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f971259c640 1 -- 172.21.15.148:0/1857443671 >> v1:172.21.15.148:6789/0 conn(0x7f97140aec20 legacy=0x7f971406acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f971259c640 1 -- 172.21.15.148:0/1857443671 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97141621d0 con 0x7f9714161df0 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7fbd8a001640 1 -- 172.21.15.148:0/809878703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd840cfdb0 msgr2=0x7fbd8406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7fbd8a001640 1 --2- 172.21.15.148:0/809878703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd840cfdb0 0x7fbd8406ecb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009870 tx=0x7fbd7802b880 comp rx=0 tx=0).stop 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f47d159a640 1 -- 172.21.15.148:0/99039390 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c402d670 con 0x7f47cc0554f0 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f041ffff640 1 --2- 172.21.15.148:0/650439708 >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 0x7f0418053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f0424fe7640 1 -- 172.21.15.148:0/650439708 >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 msgr2=0x7f0418053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f0424fe7640 1 --2- 172.21.15.148:0/650439708 >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 0x7f0418053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f0424fe7640 1 -- 172.21.15.148:0/650439708 >> v1:172.21.15.148:6789/0 conn(0x7f0418054130 legacy=0x7f04180e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f0424fe7640 1 -- 172.21.15.148:0/650439708 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418058b30 con 0x7f04180e30c0 2024-03-04T12:12:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f041ffff640 1 --2- 172.21.15.148:0/650439708 >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 0x7f0418053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f47d159a640 1 -- 172.21.15.148:0/99039390 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c402d810 con 0x7f47cc0554f0 2024-03-04T12:12:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f468402c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46840eacf0 0x7f4684185d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840547b0 con 0x7f46840069d0 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684050190 con 0x7f4684005e80 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468b59e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4684054930 con 0x7f46840eacf0 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38236/0 (socket says 172.21.15.148:38236) 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468a59c640 1 -- 172.21.15.148:0/353495704 learned_addr learned my addr 172.21.15.148:0/353495704 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7f468ad9d640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46840eacf0 0x7f4684185d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.437+0000 7ff3f3fff640 1 --2- 172.21.15.148:0/838909059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f4071890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40d0510 tx=0x7ff3e802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb08508f31bd9cc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7fbd8a001640 1 -- 172.21.15.148:0/809878703 shutdown_connections 2024-03-04T12:12:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7fbd8a001640 1 --2- 172.21.15.148:0/809878703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd840cfdb0 0x7fbd8406ecb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7fbd8a001640 1 -- 172.21.15.148:0/809878703 >> 172.21.15.148:0/809878703 conn(0x7fbd8406acd0 msgr2=0x7fbd8406b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7ff3f27fc640 1 -- 172.21.15.148:0/838909059 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80317c0 con 0x7ff3f415b510 2024-03-04T12:12:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7fbd8a001640 1 -- 172.21.15.148:0/809878703 shutdown_connections 2024-03-04T12:12:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7ff3f27fc640 1 -- 172.21.15.148:0/838909059 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8037070 con 0x7ff3f415b510 2024-03-04T12:12:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7ff3f27fc640 1 -- 172.21.15.148:0/838909059 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8031b80 con 0x7ff3f415b510 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 -- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46840eacf0 msgr2=0x7f4684185d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46840eacf0 0x7f4684185d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 -- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 msgr2=0x7f468402c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f468402c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 -- 172.21.15.148:0/353495704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468402cef0 con 0x7f4684005e80 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f0424fe7640 1 --2- 172.21.15.148:0/650439708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 0x7f0418057d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04140097a0 tx=0x7f041402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4208a0ca00c722b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7fbd8a001640 1 -- 172.21.15.148:0/809878703 wait complete. 2024-03-04T12:12:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d159a640 1 -- 172.21.15.148:0/99039390 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c402dbe0 con 0x7f47cc0554f0 2024-03-04T12:12:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f468a59c640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c3c0 tx=0x7f467c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 -- 172.21.15.148:0/99039390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0554f0 msgr2=0x7f47cc053500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 --2- 172.21.15.148:0/99039390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0554f0 0x7f47cc053500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009af0 tx=0x7f47c402bac0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f041f7fe640 1 -- 172.21.15.148:0/650439708 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414037070 con 0x7f04180e30c0 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f041f7fe640 1 -- 172.21.15.148:0/650439708 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414030710 con 0x7f04180e30c0 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 -- 172.21.15.148:0/99039390 shutdown_connections 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 --2- 172.21.15.148:0/99039390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0554f0 0x7f47cc053500 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 --2- 172.21.15.148:0/99039390 >> v2:172.21.15.191:3300/0 conn(0x7f47cc0e75e0 0x7f47cc03bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 -- 172.21.15.148:0/99039390 >> 172.21.15.148:0/99039390 conn(0x7f47cc03aa30 msgr2=0x7f47cc03ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c017070 con 0x7f4684005e80 2024-03-04T12:12:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0052f0 con 0x7f4684005e80 2024-03-04T12:12:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c003830 con 0x7f4684005e80 2024-03-04T12:12:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 -- 172.21.15.148:0/99039390 shutdown_connections 2024-03-04T12:12:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f971259c640 1 --2- 172.21.15.148:0/1857443671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f971406ea60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97140725a0 tx=0x7f970802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407b19de21d35ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f041f7fe640 1 -- 172.21.15.148:0/650439708 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414030ae0 con 0x7f04180e30c0 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f971159a640 1 -- 172.21.15.148:0/1857443671 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708034070 con 0x7f9714161df0 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f971159a640 1 -- 172.21.15.148:0/1857443671 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9708030610 con 0x7f9714161df0 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f971159a640 1 -- 172.21.15.148:0/1857443671 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080309e0 con 0x7f9714161df0 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 -- 172.21.15.148:0/650439708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 msgr2=0x7f0418057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 --2- 172.21.15.148:0/650439708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 0x7f0418057d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04140097a0 tx=0x7f041402b790 comp rx=0 tx=0).stop 2024-03-04T12:12:21.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.438+0000 7f47d359e640 1 -- 172.21.15.148:0/99039390 wait complete. 2024-03-04T12:12:21.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 -- 172.21.15.148:0/650439708 shutdown_connections 2024-03-04T12:12:21.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 --2- 172.21.15.148:0/650439708 >> v2:172.21.15.148:3300/0 conn(0x7f041803e840 0x7f0418053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 --2- 172.21.15.148:0/650439708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04180e30c0 0x7f0418057d90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7fbd8a001640 1 Processor -- start 2024-03-04T12:12:21.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.439+0000 7f0425fe9640 1 -- 172.21.15.148:0/650439708 >> 172.21.15.148:0/650439708 conn(0x7f04180559e0 msgr2=0x7f0418055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468402d1d0 con 0x7f4684005e80 2024-03-04T12:12:21.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468402d790 con 0x7f4684005e80 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 -- 172.21.15.148:0/838909059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 msgr2=0x7ff3f4071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/838909059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f4071890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40d0510 tx=0x7ff3e802b860 comp rx=0 tx=0).stop 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 -- 172.21.15.148:0/838909059 shutdown_connections 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/838909059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f4071890 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/838909059 >> v2:172.21.15.191:3300/0 conn(0x7ff3f406e1c0 0x7ff3f4071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 -- 172.21.15.148:0/838909059 >> 172.21.15.148:0/838909059 conn(0x7ff3f406acd0 msgr2=0x7ff3f406b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 -- 172.21.15.148:0/838909059 shutdown_connections 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7ff3f9e27640 1 -- 172.21.15.148:0/838909059 wait complete. 2024-03-04T12:12:21.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7f47d359e640 1 Processor -- start 2024-03-04T12:12:21.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f0425fe9640 1 -- 172.21.15.148:0/650439708 shutdown_connections 2024-03-04T12:12:21.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7f47d359e640 1 -- start start 2024-03-04T12:12:21.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.440+0000 7fbd8a001640 1 -- start start 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 -- 172.21.15.148:0/1857443671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 msgr2=0x7f971406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 --2- 172.21.15.148:0/1857443671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f971406ea60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97140725a0 tx=0x7f970802b960 comp rx=0 tx=0).stop 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd840cfdb0 0x7fbd8415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 0x7fbd8406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8406f1f0 con 0x7fbd84144e00 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84160390 con 0x7fbd841447d0 2024-03-04T12:12:21.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd8a001640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cc790 con 0x7fbd840cfdb0 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 -- 172.21.15.148:0/1857443671 shutdown_connections 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 --2- 172.21.15.148:0/1857443671 >> v2:172.21.15.148:3300/0 conn(0x7f971406b3e0 0x7f97140b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 --2- 172.21.15.148:0/1857443671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f971406ea60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 -- 172.21.15.148:0/1857443671 >> 172.21.15.148:0/1857443671 conn(0x7f97140b1130 msgr2=0x7f97140b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684005380 con 0x7f4684005e80 2024-03-04T12:12:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38248/0 (socket says 172.21.15.148:38248) 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd82ffd640 1 -- 172.21.15.148:0/366631745 learned_addr learned my addr 172.21.15.148:0/366631745 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7fbd837fe640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd840cfdb0 0x7fbd8415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7fbd82ffd640 1 -- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd840cfdb0 msgr2=0x7fbd8415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7fbd82ffd640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd840cfdb0 0x7fbd8415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7fbd82ffd640 1 -- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 msgr2=0x7fbd8406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7fbd83fff640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 0x7fbd8406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7fbd82ffd640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 0x7fbd8406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7fbd82ffd640 1 -- 172.21.15.148:0/366631745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd841f9490 con 0x7fbd841447d0 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7fbd82ffd640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd780090b0 tx=0x7fbd780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7fbd83fff640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 0x7fbd8406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47cc0554f0 0x7f47cc181cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47cc0e75e0 0x7f47cc00d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc057fb0 con 0x7f47cc056db0 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0064f0 con 0x7f47cc0e75e0 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d359e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0e79c0 con 0x7f47cc0554f0 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40170/0 (socket says 172.21.15.148:40170) 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 -- 172.21.15.148:0/2986079561 learned_addr learned my addr 172.21.15.148:0/2986079561 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d2d9d640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47cc0e75e0 0x7f47cc00d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d259c640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47cc0554f0 0x7f47cc181cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 -- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47cc0554f0 msgr2=0x7f47cc181cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47cc0554f0 0x7f47cc181cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 -- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47cc0e75e0 msgr2=0x7f47cc00d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47cc0e75e0 0x7f47cc00d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f47d1d9b640 1 -- 172.21.15.148:0/2986079561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc182200 con 0x7f47cc056db0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f47d1d9b640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009af0 tx=0x7f47c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4003810 con 0x7f47cc056db0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4045070 con 0x7f47cc056db0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4003bb0 con 0x7f47cc056db0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc0e22e0 con 0x7f47cc056db0 2024-03-04T12:12:21.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f0425fe9640 1 -- 172.21.15.148:0/650439708 wait complete. 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f467c01a070 con 0x7f4684005e80 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f467affd640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 0x7f466c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f467c089870 con 0x7f4684005e80 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f4689d9b640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 0x7f466c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.443+0000 7f4689d9b640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 0x7f466c06da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f468c000b60 tx=0x7f468c068aa0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7ff3f9e27640 1 Processor -- start 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.442+0000 7ff3f9e27640 1 -- start start 2024-03-04T12:12:21.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 0x7ff3f40a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 -- 172.21.15.148:0/1857443671 shutdown_connections 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f40a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f406b7c0 con 0x7ff3f415b510 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f406ef60 con 0x7ff3f406e1c0 2024-03-04T12:12:21.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f9e27640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f406b940 con 0x7ff3f415a700 2024-03-04T12:12:21.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.441+0000 7f9718cd1640 1 -- 172.21.15.148:0/1857443671 wait complete. 2024-03-04T12:12:21.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 0x7ff3f40a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38266/0 (socket says 172.21.15.148:38266) 2024-03-04T12:12:21.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7f9718cd1640 1 Processor -- start 2024-03-04T12:12:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f37fe640 1 -- 172.21.15.148:0/2137085440 learned_addr learned my addr 172.21.15.148:0/2137085440 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7f9718cd1640 1 -- start start 2024-03-04T12:12:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f3fff640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f40a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc0e2b80 con 0x7f47cc056db0 2024-03-04T12:12:21.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 -- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 msgr2=0x7ff3f41effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7f0425fe9640 1 Processor -- start 2024-03-04T12:12:21.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 -- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 msgr2=0x7ff3f40a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f40a9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7f0425fe9640 1 -- start start 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f2ffd640 1 -- 172.21.15.148:0/2137085440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40cf100 con 0x7ff3f415a700 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.444+0000 7ff3f37fe640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7ff3f2ffd640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 0x7ff3f40a9860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4015020 con 0x7ff3f415a700 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f47c403f070 con 0x7f47cc056db0 2024-03-04T12:12:21.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4011830 con 0x7ff3f415a700 2024-03-04T12:12:21.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4011c40 con 0x7ff3f415a700 2024-03-04T12:12:21.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7f9718cd1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 0x7f97140a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f40cf3a0 con 0x7ff3f415a700 2024-03-04T12:12:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.445+0000 7f9718cd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 0x7f971415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9718cd1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f40cf960 con 0x7ff3f415a700 2024-03-04T12:12:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9718cd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971406e260 con 0x7f97140aec20 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9718cd1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971406df80 con 0x7f9714161df0 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9718cd1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971406e540 con 0x7f971406b3e0 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f47caffd640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc0e0170 con 0x7f47cc056db0 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f47c40ac3c0 con 0x7f47cc056db0 2024-03-04T12:12:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f47d259c640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f041803e840 0x7f04180d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180e30c0 0x7f041816ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04180e34a0 con 0x7f04180e30c0 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041803bfc0 con 0x7f0418054130 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f0425fe9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04180078e0 con 0x7f041803e840 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f041ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38290/0 (socket says 172.21.15.148:38290) 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 -- 172.21.15.148:0/1332669699 learned_addr learned my addr 172.21.15.148:0/1332669699 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38274/0 (socket says 172.21.15.148:38274) 2024-03-04T12:12:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9712d9d640 1 -- 172.21.15.148:0/4238852804 learned_addr learned my addr 172.21.15.148:0/4238852804 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.446+0000 7f9711d9b640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 0x7f971415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f0424fe7640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f041803e840 0x7f04180d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f971259c640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 0x7f97140a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78032880 con 0x7fbd841447d0 2024-03-04T12:12:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f04257e8640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180e30c0 0x7f041816ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78032a20 con 0x7fbd841447d0 2024-03-04T12:12:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78032880 con 0x7fbd841447d0 2024-03-04T12:12:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f4074f60 con 0x7ff3f415a700 2024-03-04T12:12:21.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd841f9770 con 0x7fbd841447d0 2024-03-04T12:12:21.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd840af390 con 0x7fbd841447d0 2024-03-04T12:12:21.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 -- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f041803e840 msgr2=0x7f04180d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f041803e840 0x7f04180d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f467c057d40 con 0x7f4684005e80 2024-03-04T12:12:21.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 -- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180e30c0 msgr2=0x7f041816ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180e30c0 0x7f041816ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f041ffff640 1 -- 172.21.15.148:0/1332669699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180d39d0 con 0x7f0418054130 2024-03-04T12:12:21.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f47d259c640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 0x7f47b406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0079a0 tx=0x7f47bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.447+0000 7f9712d9d640 1 -- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 msgr2=0x7f97140a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9712d9d640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 0x7f97140a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9712d9d640 1 -- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 msgr2=0x7f971415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9712d9d640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 0x7f971415cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9712d9d640 1 -- 172.21.15.148:0/4238852804 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714201890 con 0x7f9714161df0 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9711d9b640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 0x7f971415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9712d9d640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b9a0 tx=0x7f970c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f971259c640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 0x7f97140a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:21.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff3e4018020 con 0x7ff3f415a700 2024-03-04T12:12:21.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7ff3dbfff640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 0x7ff3e006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.448+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c019070 con 0x7f9714161df0 2024-03-04T12:12:21.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3e408a310 con 0x7ff3f415a700 2024-03-04T12:12:21.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c00cbc0 con 0x7f9714161df0 2024-03-04T12:12:21.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7f9718cd1640 1 -- 172.21.15.148:0/4238852804 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9714201b70 con 0x7f9714161df0 2024-03-04T12:12:21.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7f9718cd1640 1 -- 172.21.15.148:0/4238852804 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9714202130 con 0x7f9714161df0 2024-03-04T12:12:21.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd64005290 con 0x7fbd841447d0 2024-03-04T12:12:21.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7ff3f37fe640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 0x7ff3e006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7f041ffff640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0420050300 tx=0x7f0420050110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.449+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042006ce60 con 0x7f0418054130 2024-03-04T12:12:21.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04180d4800 con 0x7f0418054130 2024-03-04T12:12:21.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7ff3f37fe640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 0x7ff3e006da60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec002230 tx=0x7ff3ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04180d3f60 con 0x7f0418054130 2024-03-04T12:12:21.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042008e070 con 0x7f0418054130 2024-03-04T12:12:21.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420073f70 con 0x7f0418054130 2024-03-04T12:12:21.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.451+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9714202610 con 0x7f9714161df0 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbd78032bd0 con 0x7fbd841447d0 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.450+0000 7fbd6bfff640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 0x7fbd7006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.451+0000 7fbd837fe640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 0x7fbd7006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.451+0000 7fbd837fe640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 0x7fbd7006dd90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd74004ef0 tx=0x7fbd74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.451+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbd78032680 con 0x7fbd841447d0 2024-03-04T12:12:21.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.451+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0400005290 con 0x7f0418054130 2024-03-04T12:12:21.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.452+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c014440 con 0x7f9714161df0 2024-03-04T12:12:21.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.452+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f970c01c070 con 0x7f9714161df0 2024-03-04T12:12:21.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.452+0000 7f9702ffd640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 0x7f96f406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.453+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f970c08a130 con 0x7f9714161df0 2024-03-04T12:12:21.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.453+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0420093470 con 0x7f0418054130 2024-03-04T12:12:21.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.453+0000 7f971259c640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 0x7f96f406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.453+0000 7f041d7fa640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 0x7f040406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:21.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.453+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0420104df0 con 0x7f0418054130 2024-03-04T12:12:21.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.454+0000 7f0424fe7640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 0x7f040406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:21.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.454+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3e401f020 con 0x7ff3f415a700 2024-03-04T12:12:21.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.455+0000 7f971259c640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 0x7f96f406d9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9708036000 tx=0x7f9708008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.455+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47cc0e0170 con 0x7f47cc056db0 2024-03-04T12:12:21.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.455+0000 7f0424fe7640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 0x7f040406da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0414034000 tx=0x7f041402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:21.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.456+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd78040070 con 0x7fbd841447d0 2024-03-04T12:12:21.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.461+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9714202610 con 0x7f9714161df0 2024-03-04T12:12:21.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.466+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04200d32c0 con 0x7f0418054130 2024-03-04T12:12:21.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.772+0000 7f5b2875b640 1 -- 172.21.15.148:0/3578293213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b200b00e0 con 0x7f5b200aefb0 2024-03-04T12:12:21.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.773+0000 7f5b16ffd640 1 -- 172.21.15.148:0/3578293213 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b1c058cf0 con 0x7f5b200aefb0 2024-03-04T12:12:21.775 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 msgr2=0x7f5b0406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 0x7f5b0406da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b100015d0 tx=0x7f5b10007a40 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 msgr2=0x7f5b202155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 0x7f5b202155c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00c3e0 tx=0x7f5b1c00c8a0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 shutdown_connections 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5b0406b5a0 0x7f5b0406da60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.777+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b2015c4a0 0x7f5b20164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.778+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b200aefb0 0x7f5b202155c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.778+0000 7f5b14ff9640 1 --2- 172.21.15.148:0/3578293213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b20071a60 0x7f5b2016da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.778+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 >> 172.21.15.148:0/3578293213 conn(0x7f5b2006acd0 msgr2=0x7f5b2006e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.779+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 shutdown_connections 2024-03-04T12:12:21.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.779+0000 7f5b14ff9640 1 -- 172.21.15.148:0/3578293213 wait complete. 2024-03-04T12:12:21.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-03-04T12:12:21.798 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.820+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0b54074f60 con 0x7f0b3c0039b0 2024-03-04T12:12:21.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.820+0000 7f0b42ffd640 1 -- 172.21.15.148:0/2358750287 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b4407a6a0 con 0x7f0b3c0039b0 2024-03-04T12:12:21.822 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 msgr2=0x7f0b3406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 0x7f0b3406da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b540b2cd0 tx=0x7f0b48008040 comp rx=0 tx=0).stop 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 msgr2=0x7f0b540c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b4402c440 tx=0x7f0b44004730 comp rx=0 tx=0).stop 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 shutdown_connections 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b3406b5a0 0x7f0b3406da60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b540b43f0 0x7f0b541f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b5414a760 0x7f0b540b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 --2- 172.21.15.148:0/2358750287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b3c0039b0 0x7f0b540c9440 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b4402c440 tx=0x7f0b44004730 comp rx=0 tx=0).stop 2024-03-04T12:12:21.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.823+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 >> 172.21.15.148:0/2358750287 conn(0x7f0b540d0470 msgr2=0x7f0b5406c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.825+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 shutdown_connections 2024-03-04T12:12:21.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.825+0000 7f0b40ff9640 1 -- 172.21.15.148:0/2358750287 wait complete. 2024-03-04T12:12:21.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.828+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0400005580 con 0x7f0418054130 2024-03-04T12:12:21.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.828+0000 7f041d7fa640 1 -- 172.21.15.148:0/1332669699 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04200d7360 con 0x7f0418054130 2024-03-04T12:12:21.830 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 msgr2=0x7f040406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 0x7f040406da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0414034000 tx=0x7f041402f000 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 msgr2=0x7f04180d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0420050300 tx=0x7f0420050110 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 shutdown_connections 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f040406b5a0 0x7f040406da60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180e30c0 0x7f041816ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418054130 0x7f04180d3490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 --2- 172.21.15.148:0/1332669699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f041803e840 0x7f04180d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 >> 172.21.15.148:0/1332669699 conn(0x7f04180559e0 msgr2=0x7f04180d7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 shutdown_connections 2024-03-04T12:12:21.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.831+0000 7f0425fe9640 1 -- 172.21.15.148:0/1332669699 wait complete. 2024-03-04T12:12:21.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-03-04T12:12:21.844 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-03-04T12:12:21.845 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.851+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f47cc005390 con 0x7f47cc056db0 2024-03-04T12:12:21.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.851+0000 7f47caffd640 1 -- 172.21.15.148:0/2986079561 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47cc182200 con 0x7f47cc056db0 2024-03-04T12:12:21.853 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.853+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 msgr2=0x7f47b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.853+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 0x7f47b406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0079a0 tx=0x7f47bc008040 comp rx=0 tx=0).stop 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.853+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 msgr2=0x7f47cc0e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.853+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009af0 tx=0x7f47c40046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 shutdown_connections 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47cc0e75e0 0x7f47cc00d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc056db0 0x7f47cc0e3380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 --2- 172.21.15.148:0/2986079561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47cc0554f0 0x7f47cc181cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 >> 172.21.15.148:0/2986079561 conn(0x7f47cc03aa30 msgr2=0x7f47cc006cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 shutdown_connections 2024-03-04T12:12:21.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.854+0000 7f47d359e640 1 -- 172.21.15.148:0/2986079561 wait complete. 2024-03-04T12:12:21.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.861+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4684047590 con 0x7f4684005e80 2024-03-04T12:12:21.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.861+0000 7f467affd640 1 -- 172.21.15.148:0/353495704 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f467c05bde0 con 0x7f4684005e80 2024-03-04T12:12:21.863 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-03-04T12:12:21.867 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 msgr2=0x7f466c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 0x7f466c06da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f468c000b60 tx=0x7f468c068aa0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 msgr2=0x7f468402c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c3c0 tx=0x7f467c00c880 comp rx=0 tx=0).stop 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 shutdown_connections 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f466c06b5a0 0x7f466c06da60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46840eacf0 0x7f4684185d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46840069d0 0x7f468402c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 --2- 172.21.15.148:0/353495704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4684005e80 0x7f468402c180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 >> 172.21.15.148:0/353495704 conn(0x7f4684009fa0 msgr2=0x7f4684004c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 shutdown_connections 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.865+0000 7f468b59e640 1 -- 172.21.15.148:0/353495704 wait complete. 2024-03-04T12:12:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.867+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff3f40a9640 con 0x7ff3f415a700 2024-03-04T12:12:21.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.867+0000 7ff3dbfff640 1 -- 172.21.15.148:0/2137085440 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3e40587e0 con 0x7ff3f415a700 2024-03-04T12:12:21.869 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.868+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbd64005850 con 0x7fbd841447d0 2024-03-04T12:12:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.868+0000 7fbd6bfff640 1 -- 172.21.15.148:0/366631745 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd7807a660 con 0x7fbd841447d0 2024-03-04T12:12:21.870 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.869+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 msgr2=0x7ff3e006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.869+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 0x7ff3e006da60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec002230 tx=0x7ff3ec009040 comp rx=0 tx=0).stop 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.869+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 msgr2=0x7ff3f40a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.869+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 0x7ff3f40a9860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e400b870 comp rx=0 tx=0).stop 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 shutdown_connections 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff3e006b5a0 0x7ff3e006da60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f415b510 0x7ff3f40a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3f415a700 0x7ff3f40a9860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 --2- 172.21.15.148:0/2137085440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f406e1c0 0x7ff3f41effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 >> 172.21.15.148:0/2137085440 conn(0x7ff3f406acd0 msgr2=0x7ff3f415c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 shutdown_connections 2024-03-04T12:12:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.870+0000 7ff3f9e27640 1 -- 172.21.15.148:0/2137085440 wait complete. 2024-03-04T12:12:21.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 msgr2=0x7fbd7006dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 0x7fbd7006dd90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd74004ef0 tx=0x7fbd74009040 comp rx=0 tx=0).stop 2024-03-04T12:12:21.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 msgr2=0x7fbd841322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd780090b0 tx=0x7fbd780046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 shutdown_connections 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbd7006b8d0 0x7fbd7006dd90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd84144e00 0x7fbd8406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd841447d0 0x7fbd841322f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 --2- 172.21.15.148:0/366631745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd840cfdb0 0x7fbd8415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 >> 172.21.15.148:0/366631745 conn(0x7fbd8406acd0 msgr2=0x7fbd840cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 shutdown_connections 2024-03-04T12:12:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.872+0000 7fbd8a001640 1 -- 172.21.15.148:0/366631745 wait complete. 2024-03-04T12:12:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.875+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f971406ec00 con 0x7f9714161df0 2024-03-04T12:12:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.876+0000 7f9702ffd640 1 -- 172.21.15.148:0/4238852804 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f970c0587b0 con 0x7f9714161df0 2024-03-04T12:12:21.877 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-04T12:12:21.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-03-04T12:12:21.880 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 msgr2=0x7f96f406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 0x7f96f406d9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9708036000 tx=0x7f9708008030 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 msgr2=0x7f9714200f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b9a0 tx=0x7f970c00be60 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 shutdown_connections 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f96f406b520 0x7f96f406d9e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9714161df0 0x7f9714200f20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140aec20 0x7f971415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 --2- 172.21.15.148:0/4238852804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971406b3e0 0x7f97140a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 >> 172.21.15.148:0/4238852804 conn(0x7f97140b1130 msgr2=0x7f97140b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 shutdown_connections 2024-03-04T12:12:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:21.880+0000 7f9700ff9640 1 -- 172.21.15.148:0/4238852804 wait complete. 2024-03-04T12:12:21.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-03-04T12:12:21.885 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-03-04T12:12:21.891 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-03-04T12:12:21.894 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:21.894 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-04T12:12:21.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:12:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.034+0000 7fce0e812640 1 Processor -- start 2024-03-04T12:12:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 -- start start 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce080b2e60 0x7fce080b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce080b5470 0x7fce080b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0815e1e0 con 0x7fce0815de00 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080b98c0 con 0x7fce080b2e60 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce0e812640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080b32c0 con 0x7fce080b5470 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce077fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce080b2e60 0x7fce080b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce077fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce080b2e60 0x7fce080b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40198/0 (socket says 172.21.15.148:40198) 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.035+0000 7fce077fe640 1 -- 172.21.15.148:0/1671892488 learned_addr learned my addr 172.21.15.148:0/1671892488 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce0cdaf640 1 --2- 172.21.15.148:0/1671892488 >> v2:172.21.15.191:3300/0 conn(0x7fce080b5470 0x7fce080b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce077fe640 1 -- 172.21.15.148:0/1671892488 >> v2:172.21.15.191:3300/0 conn(0x7fce080b5470 msgr2=0x7fce080b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce077fe640 1 --2- 172.21.15.148:0/1671892488 >> v2:172.21.15.191:3300/0 conn(0x7fce080b5470 0x7fce080b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce077fe640 1 -- 172.21.15.148:0/1671892488 >> v1:172.21.15.191:6789/0 conn(0x7fce0815de00 legacy=0x7fce080b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce077fe640 1 -- 172.21.15.148:0/1671892488 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce080d1c10 con 0x7fce080b2e60 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.036+0000 7fce077fe640 1 --2- 172.21.15.148:0/1671892488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b2e60 0x7fce080b9380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce080b2d40 tx=0x7fcdfc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca4586dc0c4fa8d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce06ffd640 1 -- 172.21.15.148:0/1671892488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc035020 con 0x7fce080b2e60 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce06ffd640 1 -- 172.21.15.148:0/1671892488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc031870 con 0x7fce080b2e60 2024-03-04T12:12:22.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce06ffd640 1 -- 172.21.15.148:0/1671892488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc031c40 con 0x7fce080b2e60 2024-03-04T12:12:22.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 -- 172.21.15.148:0/1671892488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b2e60 msgr2=0x7fce080b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 --2- 172.21.15.148:0/1671892488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b2e60 0x7fce080b9380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce080b2d40 tx=0x7fcdfc02b910 comp rx=0 tx=0).stop 2024-03-04T12:12:22.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 -- 172.21.15.148:0/1671892488 shutdown_connections 2024-03-04T12:12:22.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 --2- 172.21.15.148:0/1671892488 >> v2:172.21.15.191:3300/0 conn(0x7fce080b5470 0x7fce080b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 --2- 172.21.15.148:0/1671892488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b2e60 0x7fce080b9380 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 -- 172.21.15.148:0/1671892488 >> 172.21.15.148:0/1671892488 conn(0x7fce080d0470 msgr2=0x7fce080d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 -- 172.21.15.148:0/1671892488 shutdown_connections 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.037+0000 7fce0e812640 1 -- 172.21.15.148:0/1671892488 wait complete. 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.038+0000 7fce0e812640 1 Processor -- start 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 -- start start 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce080b2e60 0x7fce080c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce0815de00 0x7fce0814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce080d21d0 con 0x7fce080b5470 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce080d2820 con 0x7fce0815de00 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce0e812640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080d24b0 con 0x7fce080b2e60 2024-03-04T12:12:22.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40202/0 (socket says 172.21.15.148:40202) 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.039+0000 7fce077fe640 1 -- 172.21.15.148:0/1678169287 learned_addr learned my addr 172.21.15.148:0/1678169287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 -- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce080b2e60 msgr2=0x7fce080c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce0cdaf640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce0815de00 0x7fce0814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce07fff640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce080b2e60 0x7fce080c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce080b2e60 0x7fce080c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 -- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce0815de00 msgr2=0x7fce0814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce0815de00 0x7fce0814ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 -- 172.21.15.148:0/1678169287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08160980 con 0x7fce080b5470 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce077fe640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc03ad60 con 0x7fce080b5470 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0443f0 con 0x7fce080b5470 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce080c8630 con 0x7fce080b5470 2024-03-04T12:12:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0447e0 con 0x7fce080b5470 2024-03-04T12:12:22.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.040+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce080c8bf0 con 0x7fce080b5470 2024-03-04T12:12:22.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.041+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08074f60 con 0x7fce080b5470 2024-03-04T12:12:22.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.041+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcdfc04b060 con 0x7fce080b5470 2024-03-04T12:12:22.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.042+0000 7fce04ff9640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.042+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcdfc0ac7a0 con 0x7fce080b5470 2024-03-04T12:12:22.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.042+0000 7fce07fff640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.042+0000 7fce07fff640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 0x7fcdf406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006fd0 tx=0x7fcdf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.046+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdfc03f070 con 0x7fce080b5470 2024-03-04T12:12:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.216+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce080b2ab0 con 0x7fcdf406b4d0 2024-03-04T12:12:22.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.224+0000 7fce04ff9640 1 -- 172.21.15.148:0/1678169287 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7fce080b2ab0 con 0x7fcdf406b4d0 2024-03-04T12:12:22.229 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:22.231 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 msgr2=0x7fcdf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 0x7fcdf406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006fd0 tx=0x7fcdf8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 msgr2=0x7fce08159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 shutdown_connections 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce0815de00 0x7fce0814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce080b5470 0x7fce08159690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 --2- 172.21.15.148:0/1678169287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce080b2e60 0x7fce080c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 >> 172.21.15.148:0/1678169287 conn(0x7fce080d0470 msgr2=0x7fce080b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 shutdown_connections 2024-03-04T12:12:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.227+0000 7fce0e812640 1 -- 172.21.15.148:0/1678169287 wait complete. 2024-03-04T12:12:22.240 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-04T12:12:20.719571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003262"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771745+0000","last_change":"2024-03-04T12:11:53.892196+0000","last_active":"2024-03-04T12:12:00.771745+0000","last_peered":"2024-03-04T12:12:00.771745+0000","last_clean":"2024-03-04T12:12:00.771745+0000","last_became_active":"2024-03-04T12:11:53.891878+0000","last_became_peered":"2024-03-04T12:11:53.891878+0000","last_unstale":"2024-03-04T12:12:00.771745+0000","last_undegraded":"2024-03-04T12:12:00.771745+0000","last_fullsized":"2024-03-04T12:12:00.771745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:54.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921065+0000","last_change":"2024-03-04T12:11:54.905432+0000","last_active":"2024-03-04T12:11:59.921065+0000","last_peered":"2024-03-04T12:11:59.921065+0000","last_clean":"2024-03-04T12:11:59.921065+0000","last_became_active":"2024-03-04T12:11:54.905131+0000","last_became_peered":"2024-03-04T12:11:54.905131+0000","last_unstale":"2024-03-04T12:11:59.921065+0000","last_undegraded":"2024-03-04T12:11:59.921065+0000","last_fullsized":"2024-03-04T12:11:59.921065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920122+0000","last_change":"2024-03-04T12:11:58.938162+0000","last_active":"2024-03-04T12:11:59.920122+0000","last_peered":"2024-03-04T12:11:59.920122+0000","last_clean":"2024-03-04T12:11:59.920122+0000","last_became_active":"2024-03-04T12:11:58.937622+0000","last_became_peered":"2024-03-04T12:11:58.937622+0000","last_unstale":"2024-03-04T12:11:59.920122+0000","last_undegraded":"2024-03-04T12:11:59.920122+0000","last_fullsized":"2024-03-04T12:11:59.920122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:33:55.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921215+0000","last_change":"2024-03-04T12:11:53.881535+0000","last_active":"2024-03-04T12:11:59.921215+0000","last_peered":"2024-03-04T12:11:59.921215+0000","last_clean":"2024-03-04T12:11:59.921215+0000","last_became_active":"2024-03-04T12:11:53.880591+0000","last_became_peered":"2024-03-04T12:11:53.880591+0000","last_unstale":"2024-03-04T12:11:59.921215+0000","last_undegraded":"2024-03-04T12:11:59.921215+0000","last_fullsized":"2024-03-04T12:11:59.921215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771443+0000","last_change":"2024-03-04T12:11:54.906418+0000","last_active":"2024-03-04T12:12:00.771443+0000","last_peered":"2024-03-04T12:12:00.771443+0000","last_clean":"2024-03-04T12:12:00.771443+0000","last_became_active":"2024-03-04T12:11:54.906146+0000","last_became_peered":"2024-03-04T12:11:54.906146+0000","last_unstale":"2024-03-04T12:12:00.771443+0000","last_undegraded":"2024-03-04T12:12:00.771443+0000","last_fullsized":"2024-03-04T12:12:00.771443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771197+0000","last_change":"2024-03-04T12:11:58.945630+0000","last_active":"2024-03-04T12:12:00.771197+0000","last_peered":"2024-03-04T12:12:00.771197+0000","last_clean":"2024-03-04T12:12:00.771197+0000","last_became_active":"2024-03-04T12:11:58.945230+0000","last_became_peered":"2024-03-04T12:11:58.945230+0000","last_unstale":"2024-03-04T12:12:00.771197+0000","last_undegraded":"2024-03-04T12:12:00.771197+0000","last_fullsized":"2024-03-04T12:12:00.771197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:50.652152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.895431+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.895122+0000","last_became_peered":"2024-03-04T12:11:53.895122+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:54.330076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771997+0000","last_change":"2024-03-04T12:11:54.902410+0000","last_active":"2024-03-04T12:12:00.771997+0000","last_peered":"2024-03-04T12:12:00.771997+0000","last_clean":"2024-03-04T12:12:00.771997+0000","last_became_active":"2024-03-04T12:11:54.902131+0000","last_became_peered":"2024-03-04T12:11:54.902131+0000","last_unstale":"2024-03-04T12:12:00.771997+0000","last_undegraded":"2024-03-04T12:12:00.771997+0000","last_fullsized":"2024-03-04T12:12:00.771997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772908+0000","last_change":"2024-03-04T12:11:58.943323+0000","last_active":"2024-03-04T12:12:00.772908+0000","last_peered":"2024-03-04T12:12:00.772908+0000","last_clean":"2024-03-04T12:12:00.772908+0000","last_became_active":"2024-03-04T12:11:58.942878+0000","last_became_peered":"2024-03-04T12:11:58.942878+0000","last_unstale":"2024-03-04T12:12:00.772908+0000","last_undegraded":"2024-03-04T12:12:00.772908+0000","last_fullsized":"2024-03-04T12:12:00.772908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771622+0000","last_change":"2024-03-04T12:11:53.893583+0000","last_active":"2024-03-04T12:12:00.771622+0000","last_peered":"2024-03-04T12:12:00.771622+0000","last_clean":"2024-03-04T12:12:00.771622+0000","last_became_active":"2024-03-04T12:11:53.893295+0000","last_became_peered":"2024-03-04T12:11:53.893295+0000","last_unstale":"2024-03-04T12:12:00.771622+0000","last_undegraded":"2024-03-04T12:12:00.771622+0000","last_fullsized":"2024-03-04T12:12:00.771622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772748+0000","last_change":"2024-03-04T12:11:54.901339+0000","last_active":"2024-03-04T12:12:00.772748+0000","last_peered":"2024-03-04T12:12:00.772748+0000","last_clean":"2024-03-04T12:12:00.772748+0000","last_became_active":"2024-03-04T12:11:54.900914+0000","last_became_peered":"2024-03-04T12:11:54.900914+0000","last_unstale":"2024-03-04T12:12:00.772748+0000","last_undegraded":"2024-03-04T12:12:00.772748+0000","last_fullsized":"2024-03-04T12:12:00.772748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773092+0000","last_change":"2024-03-04T12:11:58.937633+0000","last_active":"2024-03-04T12:12:00.773092+0000","last_peered":"2024-03-04T12:12:00.773092+0000","last_clean":"2024-03-04T12:12:00.773092+0000","last_became_active":"2024-03-04T12:11:58.936727+0000","last_became_peered":"2024-03-04T12:11:58.936727+0000","last_unstale":"2024-03-04T12:12:00.773092+0000","last_undegraded":"2024-03-04T12:12:00.773092+0000","last_fullsized":"2024-03-04T12:12:00.773092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:03:00.255209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921104+0000","last_change":"2024-03-04T12:11:54.905529+0000","last_active":"2024-03-04T12:11:59.921104+0000","last_peered":"2024-03-04T12:11:59.921104+0000","last_clean":"2024-03-04T12:11:59.921104+0000","last_became_active":"2024-03-04T12:11:54.904590+0000","last_became_peered":"2024-03-04T12:11:54.904590+0000","last_unstale":"2024-03-04T12:11:59.921104+0000","last_undegraded":"2024-03-04T12:11:59.921104+0000","last_fullsized":"2024-03-04T12:11:59.921104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772191+0000","last_change":"2024-03-04T12:11:53.892967+0000","last_active":"2024-03-04T12:12:00.772191+0000","last_peered":"2024-03-04T12:12:00.772191+0000","last_clean":"2024-03-04T12:12:00.772191+0000","last_became_active":"2024-03-04T12:11:53.892698+0000","last_became_peered":"2024-03-04T12:11:53.892698+0000","last_unstale":"2024-03-04T12:12:00.772191+0000","last_undegraded":"2024-03-04T12:12:00.772191+0000","last_fullsized":"2024-03-04T12:12:00.772191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772796+0000","last_change":"2024-03-04T12:11:58.947133+0000","last_active":"2024-03-04T12:12:00.772796+0000","last_peered":"2024-03-04T12:12:00.772796+0000","last_clean":"2024-03-04T12:12:00.772796+0000","last_became_active":"2024-03-04T12:11:58.946617+0000","last_became_peered":"2024-03-04T12:11:58.946617+0000","last_unstale":"2024-03-04T12:12:00.772796+0000","last_undegraded":"2024-03-04T12:12:00.772796+0000","last_fullsized":"2024-03-04T12:12:00.772796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920487+0000","last_change":"2024-03-04T12:11:54.896623+0000","last_active":"2024-03-04T12:11:59.920487+0000","last_peered":"2024-03-04T12:11:59.920487+0000","last_clean":"2024-03-04T12:11:59.920487+0000","last_became_active":"2024-03-04T12:11:54.896359+0000","last_became_peered":"2024-03-04T12:11:54.896359+0000","last_unstale":"2024-03-04T12:11:59.920487+0000","last_undegraded":"2024-03-04T12:11:59.920487+0000","last_fullsized":"2024-03-04T12:11:59.920487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771403+0000","last_change":"2024-03-04T12:11:53.886825+0000","last_active":"2024-03-04T12:12:00.771403+0000","last_peered":"2024-03-04T12:12:00.771403+0000","last_clean":"2024-03-04T12:12:00.771403+0000","last_became_active":"2024-03-04T12:11:53.886525+0000","last_became_peered":"2024-03-04T12:11:53.886525+0000","last_unstale":"2024-03-04T12:12:00.771403+0000","last_undegraded":"2024-03-04T12:12:00.771403+0000","last_fullsized":"2024-03-04T12:12:00.771403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772607+0000","last_change":"2024-03-04T12:11:58.946327+0000","last_active":"2024-03-04T12:12:00.772607+0000","last_peered":"2024-03-04T12:12:00.772607+0000","last_clean":"2024-03-04T12:12:00.772607+0000","last_became_active":"2024-03-04T12:11:58.945962+0000","last_became_peered":"2024-03-04T12:11:58.945962+0000","last_unstale":"2024-03-04T12:12:00.772607+0000","last_undegraded":"2024-03-04T12:12:00.772607+0000","last_fullsized":"2024-03-04T12:12:00.772607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920261+0000","last_change":"2024-03-04T12:11:54.903139+0000","last_active":"2024-03-04T12:11:59.920261+0000","last_peered":"2024-03-04T12:11:59.920261+0000","last_clean":"2024-03-04T12:11:59.920261+0000","last_became_active":"2024-03-04T12:11:54.902809+0000","last_became_peered":"2024-03-04T12:11:54.902809+0000","last_unstale":"2024-03-04T12:11:59.920261+0000","last_undegraded":"2024-03-04T12:11:59.920261+0000","last_fullsized":"2024-03-04T12:11:59.920261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770887+0000","last_change":"2024-03-04T12:11:53.889802+0000","last_active":"2024-03-04T12:12:00.770887+0000","last_peered":"2024-03-04T12:12:00.770887+0000","last_clean":"2024-03-04T12:12:00.770887+0000","last_became_active":"2024-03-04T12:11:53.889550+0000","last_became_peered":"2024-03-04T12:11:53.889550+0000","last_unstale":"2024-03-04T12:12:00.770887+0000","last_undegraded":"2024-03-04T12:12:00.770887+0000","last_fullsized":"2024-03-04T12:12:00.770887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771267+0000","last_change":"2024-03-04T12:11:58.941310+0000","last_active":"2024-03-04T12:12:00.771267+0000","last_peered":"2024-03-04T12:12:00.771267+0000","last_clean":"2024-03-04T12:12:00.771267+0000","last_became_active":"2024-03-04T12:11:58.940932+0000","last_became_peered":"2024-03-04T12:11:58.940932+0000","last_unstale":"2024-03-04T12:12:00.771267+0000","last_undegraded":"2024-03-04T12:12:00.771267+0000","last_fullsized":"2024-03-04T12:12:00.771267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772213+0000","last_change":"2024-03-04T12:11:54.900654+0000","last_active":"2024-03-04T12:12:00.772213+0000","last_peered":"2024-03-04T12:12:00.772213+0000","last_clean":"2024-03-04T12:12:00.772213+0000","last_became_active":"2024-03-04T12:11:54.900311+0000","last_became_peered":"2024-03-04T12:11:54.900311+0000","last_unstale":"2024-03-04T12:12:00.772213+0000","last_undegraded":"2024-03-04T12:12:00.772213+0000","last_fullsized":"2024-03-04T12:12:00.772213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:23.687708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771071+0000","last_change":"2024-03-04T12:11:53.892220+0000","last_active":"2024-03-04T12:12:00.771071+0000","last_peered":"2024-03-04T12:12:00.771071+0000","last_clean":"2024-03-04T12:12:00.771071+0000","last_became_active":"2024-03-04T12:11:53.891859+0000","last_became_peered":"2024-03-04T12:11:53.891859+0000","last_unstale":"2024-03-04T12:12:00.771071+0000","last_undegraded":"2024-03-04T12:12:00.771071+0000","last_fullsized":"2024-03-04T12:12:00.771071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771968+0000","last_change":"2024-03-04T12:11:58.947661+0000","last_active":"2024-03-04T12:12:00.771968+0000","last_peered":"2024-03-04T12:12:00.771968+0000","last_clean":"2024-03-04T12:12:00.771968+0000","last_became_active":"2024-03-04T12:11:58.946889+0000","last_became_peered":"2024-03-04T12:11:58.946889+0000","last_unstale":"2024-03-04T12:12:00.771968+0000","last_undegraded":"2024-03-04T12:12:00.771968+0000","last_fullsized":"2024-03-04T12:12:00.771968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:48.995386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771351+0000","last_change":"2024-03-04T12:11:54.898183+0000","last_active":"2024-03-04T12:12:00.771351+0000","last_peered":"2024-03-04T12:12:00.771351+0000","last_clean":"2024-03-04T12:12:00.771351+0000","last_became_active":"2024-03-04T12:11:54.897873+0000","last_became_peered":"2024-03-04T12:11:54.897873+0000","last_unstale":"2024-03-04T12:12:00.771351+0000","last_undegraded":"2024-03-04T12:12:00.771351+0000","last_fullsized":"2024-03-04T12:12:00.771351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921618+0000","last_change":"2024-03-04T12:11:53.893093+0000","last_active":"2024-03-04T12:11:59.921618+0000","last_peered":"2024-03-04T12:11:59.921618+0000","last_clean":"2024-03-04T12:11:59.921618+0000","last_became_active":"2024-03-04T12:11:53.892821+0000","last_became_peered":"2024-03-04T12:11:53.892821+0000","last_unstale":"2024-03-04T12:11:59.921618+0000","last_undegraded":"2024-03-04T12:11:59.921618+0000","last_fullsized":"2024-03-04T12:11:59.921618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771969+0000","last_change":"2024-03-04T12:11:58.936275+0000","last_active":"2024-03-04T12:12:00.771969+0000","last_peered":"2024-03-04T12:12:00.771969+0000","last_clean":"2024-03-04T12:12:00.771969+0000","last_became_active":"2024-03-04T12:11:58.935879+0000","last_became_peered":"2024-03-04T12:11:58.935879+0000","last_unstale":"2024-03-04T12:12:00.771969+0000","last_undegraded":"2024-03-04T12:12:00.771969+0000","last_fullsized":"2024-03-04T12:12:00.771969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:57.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921287+0000","last_change":"2024-03-04T12:11:54.905996+0000","last_active":"2024-03-04T12:11:59.921287+0000","last_peered":"2024-03-04T12:11:59.921287+0000","last_clean":"2024-03-04T12:11:59.921287+0000","last_became_active":"2024-03-04T12:11:54.905164+0000","last_became_peered":"2024-03-04T12:11:54.905164+0000","last_unstale":"2024-03-04T12:11:59.921287+0000","last_undegraded":"2024-03-04T12:11:59.921287+0000","last_fullsized":"2024-03-04T12:11:59.921287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771364+0000","last_change":"2024-03-04T12:11:53.889023+0000","last_active":"2024-03-04T12:12:00.771364+0000","last_peered":"2024-03-04T12:12:00.771364+0000","last_clean":"2024-03-04T12:12:00.771364+0000","last_became_active":"2024-03-04T12:11:53.888732+0000","last_became_peered":"2024-03-04T12:11:53.888732+0000","last_unstale":"2024-03-04T12:12:00.771364+0000","last_undegraded":"2024-03-04T12:12:00.771364+0000","last_fullsized":"2024-03-04T12:12:00.771364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920041+0000","last_change":"2024-03-04T12:11:58.933844+0000","last_active":"2024-03-04T12:11:59.920041+0000","last_peered":"2024-03-04T12:11:59.920041+0000","last_clean":"2024-03-04T12:11:59.920041+0000","last_became_active":"2024-03-04T12:11:58.933456+0000","last_became_peered":"2024-03-04T12:11:58.933456+0000","last_unstale":"2024-03-04T12:11:59.920041+0000","last_undegraded":"2024-03-04T12:11:59.920041+0000","last_fullsized":"2024-03-04T12:11:59.920041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771789+0000","last_change":"2024-03-04T12:11:54.900251+0000","last_active":"2024-03-04T12:12:00.771789+0000","last_peered":"2024-03-04T12:12:00.771789+0000","last_clean":"2024-03-04T12:12:00.771789+0000","last_became_active":"2024-03-04T12:11:54.899988+0000","last_became_peered":"2024-03-04T12:11:54.899988+0000","last_unstale":"2024-03-04T12:12:00.771789+0000","last_undegraded":"2024-03-04T12:12:00.771789+0000","last_fullsized":"2024-03-04T12:12:00.771789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771966+0000","last_change":"2024-03-04T12:11:53.893557+0000","last_active":"2024-03-04T12:12:00.771966+0000","last_peered":"2024-03-04T12:12:00.771966+0000","last_clean":"2024-03-04T12:12:00.771966+0000","last_became_active":"2024-03-04T12:11:53.893170+0000","last_became_peered":"2024-03-04T12:11:53.893170+0000","last_unstale":"2024-03-04T12:12:00.771966+0000","last_undegraded":"2024-03-04T12:12:00.771966+0000","last_fullsized":"2024-03-04T12:12:00.771966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:04.682290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770797+0000","last_change":"2024-03-04T12:11:58.944095+0000","last_active":"2024-03-04T12:12:00.770797+0000","last_peered":"2024-03-04T12:12:00.770797+0000","last_clean":"2024-03-04T12:12:00.770797+0000","last_became_active":"2024-03-04T12:11:58.943691+0000","last_became_peered":"2024-03-04T12:11:58.943691+0000","last_unstale":"2024-03-04T12:12:00.770797+0000","last_undegraded":"2024-03-04T12:12:00.770797+0000","last_fullsized":"2024-03-04T12:12:00.770797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772840+0000","last_change":"2024-03-04T12:11:54.909256+0000","last_active":"2024-03-04T12:12:00.772840+0000","last_peered":"2024-03-04T12:12:00.772840+0000","last_clean":"2024-03-04T12:12:00.772840+0000","last_became_active":"2024-03-04T12:11:54.908971+0000","last_became_peered":"2024-03-04T12:11:54.908971+0000","last_unstale":"2024-03-04T12:12:00.772840+0000","last_undegraded":"2024-03-04T12:12:00.772840+0000","last_fullsized":"2024-03-04T12:12:00.772840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771832+0000","last_change":"2024-03-04T12:11:53.881754+0000","last_active":"2024-03-04T12:12:00.771832+0000","last_peered":"2024-03-04T12:12:00.771832+0000","last_clean":"2024-03-04T12:12:00.771832+0000","last_became_active":"2024-03-04T12:11:53.881149+0000","last_became_peered":"2024-03-04T12:11:53.881149+0000","last_unstale":"2024-03-04T12:12:00.771832+0000","last_undegraded":"2024-03-04T12:12:00.771832+0000","last_fullsized":"2024-03-04T12:12:00.771832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771018+0000","last_change":"2024-03-04T12:11:58.939152+0000","last_active":"2024-03-04T12:12:00.771018+0000","last_peered":"2024-03-04T12:12:00.771018+0000","last_clean":"2024-03-04T12:12:00.771018+0000","last_became_active":"2024-03-04T12:11:58.938820+0000","last_became_peered":"2024-03-04T12:11:58.938820+0000","last_unstale":"2024-03-04T12:12:00.771018+0000","last_undegraded":"2024-03-04T12:12:00.771018+0000","last_fullsized":"2024-03-04T12:12:00.771018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773199+0000","last_change":"2024-03-04T12:11:54.893605+0000","last_active":"2024-03-04T12:12:00.773199+0000","last_peered":"2024-03-04T12:12:00.773199+0000","last_clean":"2024-03-04T12:12:00.773199+0000","last_became_active":"2024-03-04T12:11:54.893164+0000","last_became_peered":"2024-03-04T12:11:54.893164+0000","last_unstale":"2024-03-04T12:12:00.773199+0000","last_undegraded":"2024-03-04T12:12:00.773199+0000","last_fullsized":"2024-03-04T12:12:00.773199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:32.197141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.884670+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.884396+0000","last_became_peered":"2024-03-04T12:11:53.884396+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771791+0000","last_change":"2024-03-04T12:11:58.939922+0000","last_active":"2024-03-04T12:12:00.771791+0000","last_peered":"2024-03-04T12:12:00.771791+0000","last_clean":"2024-03-04T12:12:00.771791+0000","last_became_active":"2024-03-04T12:11:58.939562+0000","last_became_peered":"2024-03-04T12:11:58.939562+0000","last_unstale":"2024-03-04T12:12:00.771791+0000","last_undegraded":"2024-03-04T12:12:00.771791+0000","last_fullsized":"2024-03-04T12:12:00.771791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770827+0000","last_change":"2024-03-04T12:11:54.898108+0000","last_active":"2024-03-04T12:12:00.770827+0000","last_peered":"2024-03-04T12:12:00.770827+0000","last_clean":"2024-03-04T12:12:00.770827+0000","last_became_active":"2024-03-04T12:11:54.897832+0000","last_became_peered":"2024-03-04T12:11:54.897832+0000","last_unstale":"2024-03-04T12:12:00.770827+0000","last_undegraded":"2024-03-04T12:12:00.770827+0000","last_fullsized":"2024-03-04T12:12:00.770827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921186+0000","last_change":"2024-03-04T12:11:53.884846+0000","last_active":"2024-03-04T12:11:59.921186+0000","last_peered":"2024-03-04T12:11:59.921186+0000","last_clean":"2024-03-04T12:11:59.921186+0000","last_became_active":"2024-03-04T12:11:53.884567+0000","last_became_peered":"2024-03-04T12:11:53.884567+0000","last_unstale":"2024-03-04T12:11:59.921186+0000","last_undegraded":"2024-03-04T12:11:59.921186+0000","last_fullsized":"2024-03-04T12:11:59.921186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771888+0000","last_change":"2024-03-04T12:11:58.936068+0000","last_active":"2024-03-04T12:12:00.771888+0000","last_peered":"2024-03-04T12:12:00.771888+0000","last_clean":"2024-03-04T12:12:00.771888+0000","last_became_active":"2024-03-04T12:11:58.935721+0000","last_became_peered":"2024-03-04T12:11:58.935721+0000","last_unstale":"2024-03-04T12:12:00.771888+0000","last_undegraded":"2024-03-04T12:12:00.771888+0000","last_fullsized":"2024-03-04T12:12:00.771888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770381+0000","last_change":"2024-03-04T12:11:54.893726+0000","last_active":"2024-03-04T12:12:00.770381+0000","last_peered":"2024-03-04T12:12:00.770381+0000","last_clean":"2024-03-04T12:12:00.770381+0000","last_became_active":"2024-03-04T12:11:54.893038+0000","last_became_peered":"2024-03-04T12:11:54.893038+0000","last_unstale":"2024-03-04T12:12:00.770381+0000","last_undegraded":"2024-03-04T12:12:00.770381+0000","last_fullsized":"2024-03-04T12:12:00.770381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770859+0000","last_change":"2024-03-04T12:11:53.893383+0000","last_active":"2024-03-04T12:12:00.770859+0000","last_peered":"2024-03-04T12:12:00.770859+0000","last_clean":"2024-03-04T12:12:00.770859+0000","last_became_active":"2024-03-04T12:11:53.892630+0000","last_became_peered":"2024-03-04T12:11:53.892630+0000","last_unstale":"2024-03-04T12:12:00.770859+0000","last_undegraded":"2024-03-04T12:12:00.770859+0000","last_fullsized":"2024-03-04T12:12:00.770859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771246+0000","last_change":"2024-03-04T12:11:58.946451+0000","last_active":"2024-03-04T12:12:00.771246+0000","last_peered":"2024-03-04T12:12:00.771246+0000","last_clean":"2024-03-04T12:12:00.771246+0000","last_became_active":"2024-03-04T12:11:58.946080+0000","last_became_peered":"2024-03-04T12:11:58.946080+0000","last_unstale":"2024-03-04T12:12:00.771246+0000","last_undegraded":"2024-03-04T12:12:00.771246+0000","last_fullsized":"2024-03-04T12:12:00.771246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920426+0000","last_change":"2024-03-04T12:11:54.902080+0000","last_active":"2024-03-04T12:11:59.920426+0000","last_peered":"2024-03-04T12:11:59.920426+0000","last_clean":"2024-03-04T12:11:59.920426+0000","last_became_active":"2024-03-04T12:11:54.901382+0000","last_became_peered":"2024-03-04T12:11:54.901382+0000","last_unstale":"2024-03-04T12:11:59.920426+0000","last_undegraded":"2024-03-04T12:11:59.920426+0000","last_fullsized":"2024-03-04T12:11:59.920426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772165+0000","last_change":"2024-03-04T12:11:53.883991+0000","last_active":"2024-03-04T12:12:00.772165+0000","last_peered":"2024-03-04T12:12:00.772165+0000","last_clean":"2024-03-04T12:12:00.772165+0000","last_became_active":"2024-03-04T12:11:53.881700+0000","last_became_peered":"2024-03-04T12:11:53.881700+0000","last_unstale":"2024-03-04T12:12:00.772165+0000","last_undegraded":"2024-03-04T12:12:00.772165+0000","last_fullsized":"2024-03-04T12:12:00.772165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770976+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:12:00.770976+0000","last_peered":"2024-03-04T12:12:00.770976+0000","last_clean":"2024-03-04T12:12:00.770976+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:12:00.770976+0000","last_undegraded":"2024-03-04T12:12:00.770976+0000","last_fullsized":"2024-03-04T12:12:00.770976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921000+0000","last_change":"2024-03-04T12:11:58.939590+0000","last_active":"2024-03-04T12:11:59.921000+0000","last_peered":"2024-03-04T12:11:59.921000+0000","last_clean":"2024-03-04T12:11:59.921000+0000","last_became_active":"2024-03-04T12:11:58.939177+0000","last_became_peered":"2024-03-04T12:11:58.939177+0000","last_unstale":"2024-03-04T12:11:59.921000+0000","last_undegraded":"2024-03-04T12:11:59.921000+0000","last_fullsized":"2024-03-04T12:11:59.921000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:54.906271+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:54.905428+0000","last_became_peered":"2024-03-04T12:11:54.905428+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921284+0000","last_change":"2024-03-04T12:11:53.885617+0000","last_active":"2024-03-04T12:11:59.921284+0000","last_peered":"2024-03-04T12:11:59.921284+0000","last_clean":"2024-03-04T12:11:59.921284+0000","last_became_active":"2024-03-04T12:11:53.885329+0000","last_became_peered":"2024-03-04T12:11:53.885329+0000","last_unstale":"2024-03-04T12:11:59.921284+0000","last_undegraded":"2024-03-04T12:11:59.921284+0000","last_fullsized":"2024-03-04T12:11:59.921284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921212+0000","last_change":"2024-03-04T12:11:58.946535+0000","last_active":"2024-03-04T12:11:59.921212+0000","last_peered":"2024-03-04T12:11:59.921212+0000","last_clean":"2024-03-04T12:11:59.921212+0000","last_became_active":"2024-03-04T12:11:58.946055+0000","last_became_peered":"2024-03-04T12:11:58.946055+0000","last_unstale":"2024-03-04T12:11:59.921212+0000","last_undegraded":"2024-03-04T12:11:59.921212+0000","last_fullsized":"2024-03-04T12:11:59.921212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772109+0000","last_change":"2024-03-04T12:11:54.910474+0000","last_active":"2024-03-04T12:12:00.772109+0000","last_peered":"2024-03-04T12:12:00.772109+0000","last_clean":"2024-03-04T12:12:00.772109+0000","last_became_active":"2024-03-04T12:11:54.909806+0000","last_became_peered":"2024-03-04T12:11:54.909806+0000","last_unstale":"2024-03-04T12:12:00.772109+0000","last_undegraded":"2024-03-04T12:12:00.772109+0000","last_fullsized":"2024-03-04T12:12:00.772109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771668+0000","last_change":"2024-03-04T12:11:53.882414+0000","last_active":"2024-03-04T12:12:00.771668+0000","last_peered":"2024-03-04T12:12:00.771668+0000","last_clean":"2024-03-04T12:12:00.771668+0000","last_became_active":"2024-03-04T12:11:53.882147+0000","last_became_peered":"2024-03-04T12:11:53.882147+0000","last_unstale":"2024-03-04T12:12:00.771668+0000","last_undegraded":"2024-03-04T12:12:00.771668+0000","last_fullsized":"2024-03-04T12:12:00.771668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771019+0000","last_change":"2024-03-04T12:11:58.946384+0000","last_active":"2024-03-04T12:12:00.771019+0000","last_peered":"2024-03-04T12:12:00.771019+0000","last_clean":"2024-03-04T12:12:00.771019+0000","last_became_active":"2024-03-04T12:11:58.945989+0000","last_became_peered":"2024-03-04T12:11:58.945989+0000","last_unstale":"2024-03-04T12:12:00.771019+0000","last_undegraded":"2024-03-04T12:12:00.771019+0000","last_fullsized":"2024-03-04T12:12:00.771019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772496+0000","last_change":"2024-03-04T12:11:54.903131+0000","last_active":"2024-03-04T12:12:00.772496+0000","last_peered":"2024-03-04T12:12:00.772496+0000","last_clean":"2024-03-04T12:12:00.772496+0000","last_became_active":"2024-03-04T12:11:54.902729+0000","last_became_peered":"2024-03-04T12:11:54.902729+0000","last_unstale":"2024-03-04T12:12:00.772496+0000","last_undegraded":"2024-03-04T12:12:00.772496+0000","last_fullsized":"2024-03-04T12:12:00.772496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:43.148028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921116+0000","last_change":"2024-03-04T12:11:53.884623+0000","last_active":"2024-03-04T12:11:59.921116+0000","last_peered":"2024-03-04T12:11:59.921116+0000","last_clean":"2024-03-04T12:11:59.921116+0000","last_became_active":"2024-03-04T12:11:53.884222+0000","last_became_peered":"2024-03-04T12:11:53.884222+0000","last_unstale":"2024-03-04T12:11:59.921116+0000","last_undegraded":"2024-03-04T12:11:59.921116+0000","last_fullsized":"2024-03-04T12:11:59.921116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771150+0000","last_change":"2024-03-04T12:11:58.935385+0000","last_active":"2024-03-04T12:12:00.771150+0000","last_peered":"2024-03-04T12:12:00.771150+0000","last_clean":"2024-03-04T12:12:00.771150+0000","last_became_active":"2024-03-04T12:11:58.934991+0000","last_became_peered":"2024-03-04T12:11:58.934991+0000","last_unstale":"2024-03-04T12:12:00.771150+0000","last_undegraded":"2024-03-04T12:12:00.771150+0000","last_fullsized":"2024-03-04T12:12:00.771150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:09.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773010+0000","last_change":"2024-03-04T12:11:54.901559+0000","last_active":"2024-03-04T12:12:00.773010+0000","last_peered":"2024-03-04T12:12:00.773010+0000","last_clean":"2024-03-04T12:12:00.773010+0000","last_became_active":"2024-03-04T12:11:54.901215+0000","last_became_peered":"2024-03-04T12:11:54.901215+0000","last_unstale":"2024-03-04T12:12:00.773010+0000","last_undegraded":"2024-03-04T12:12:00.773010+0000","last_fullsized":"2024-03-04T12:12:00.773010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772226+0000","last_change":"2024-03-04T12:11:53.884589+0000","last_active":"2024-03-04T12:12:00.772226+0000","last_peered":"2024-03-04T12:12:00.772226+0000","last_clean":"2024-03-04T12:12:00.772226+0000","last_became_active":"2024-03-04T12:11:53.884279+0000","last_became_peered":"2024-03-04T12:11:53.884279+0000","last_unstale":"2024-03-04T12:12:00.772226+0000","last_undegraded":"2024-03-04T12:12:00.772226+0000","last_fullsized":"2024-03-04T12:12:00.772226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:54.931530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771590+0000","last_change":"2024-03-04T12:11:58.945665+0000","last_active":"2024-03-04T12:12:00.771590+0000","last_peered":"2024-03-04T12:12:00.771590+0000","last_clean":"2024-03-04T12:12:00.771590+0000","last_became_active":"2024-03-04T12:11:58.945297+0000","last_became_peered":"2024-03-04T12:11:58.945297+0000","last_unstale":"2024-03-04T12:12:00.771590+0000","last_undegraded":"2024-03-04T12:12:00.771590+0000","last_fullsized":"2024-03-04T12:12:00.771590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:11.565262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771183+0000","last_change":"2024-03-04T12:11:54.902251+0000","last_active":"2024-03-04T12:12:00.771183+0000","last_peered":"2024-03-04T12:12:00.771183+0000","last_clean":"2024-03-04T12:12:00.771183+0000","last_became_active":"2024-03-04T12:11:54.901942+0000","last_became_peered":"2024-03-04T12:11:54.901942+0000","last_unstale":"2024-03-04T12:12:00.771183+0000","last_undegraded":"2024-03-04T12:12:00.771183+0000","last_fullsized":"2024-03-04T12:12:00.771183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920811+0000","last_change":"2024-03-04T12:11:53.883599+0000","last_active":"2024-03-04T12:11:59.920811+0000","last_peered":"2024-03-04T12:11:59.920811+0000","last_clean":"2024-03-04T12:11:59.920811+0000","last_became_active":"2024-03-04T12:11:53.883317+0000","last_became_peered":"2024-03-04T12:11:53.883317+0000","last_unstale":"2024-03-04T12:11:59.920811+0000","last_undegraded":"2024-03-04T12:11:59.920811+0000","last_fullsized":"2024-03-04T12:11:59.920811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771562+0000","last_change":"2024-03-04T12:11:58.936436+0000","last_active":"2024-03-04T12:12:00.771562+0000","last_peered":"2024-03-04T12:12:00.771562+0000","last_clean":"2024-03-04T12:12:00.771562+0000","last_became_active":"2024-03-04T12:11:58.935593+0000","last_became_peered":"2024-03-04T12:11:58.935593+0000","last_unstale":"2024-03-04T12:12:00.771562+0000","last_undegraded":"2024-03-04T12:12:00.771562+0000","last_fullsized":"2024-03-04T12:12:00.771562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920930+0000","last_change":"2024-03-04T12:11:54.908033+0000","last_active":"2024-03-04T12:11:59.920930+0000","last_peered":"2024-03-04T12:11:59.920930+0000","last_clean":"2024-03-04T12:11:59.920930+0000","last_became_active":"2024-03-04T12:11:54.907784+0000","last_became_peered":"2024-03-04T12:11:54.907784+0000","last_unstale":"2024-03-04T12:11:59.920930+0000","last_undegraded":"2024-03-04T12:11:59.920930+0000","last_fullsized":"2024-03-04T12:11:59.920930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921086+0000","last_change":"2024-03-04T12:11:53.889166+0000","last_active":"2024-03-04T12:11:59.921086+0000","last_peered":"2024-03-04T12:11:59.921086+0000","last_clean":"2024-03-04T12:11:59.921086+0000","last_became_active":"2024-03-04T12:11:53.888906+0000","last_became_peered":"2024-03-04T12:11:53.888906+0000","last_unstale":"2024-03-04T12:11:59.921086+0000","last_undegraded":"2024-03-04T12:11:59.921086+0000","last_fullsized":"2024-03-04T12:11:59.921086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921204+0000","last_change":"2024-03-04T12:11:58.944078+0000","last_active":"2024-03-04T12:11:59.921204+0000","last_peered":"2024-03-04T12:11:59.921204+0000","last_clean":"2024-03-04T12:11:59.921204+0000","last_became_active":"2024-03-04T12:11:58.942753+0000","last_became_peered":"2024-03-04T12:11:58.942753+0000","last_unstale":"2024-03-04T12:11:59.921204+0000","last_undegraded":"2024-03-04T12:11:59.921204+0000","last_fullsized":"2024-03-04T12:11:59.921204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:35.014220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772005+0000","last_change":"2024-03-04T12:11:54.902835+0000","last_active":"2024-03-04T12:12:00.772005+0000","last_peered":"2024-03-04T12:12:00.772005+0000","last_clean":"2024-03-04T12:12:00.772005+0000","last_became_active":"2024-03-04T12:11:54.902550+0000","last_became_peered":"2024-03-04T12:11:54.902550+0000","last_unstale":"2024-03-04T12:12:00.772005+0000","last_undegraded":"2024-03-04T12:12:00.772005+0000","last_fullsized":"2024-03-04T12:12:00.772005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:53.885828+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:53.885526+0000","last_became_peered":"2024-03-04T12:11:53.885526+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920921+0000","last_change":"2024-03-04T12:11:58.943710+0000","last_active":"2024-03-04T12:11:59.920921+0000","last_peered":"2024-03-04T12:11:59.920921+0000","last_clean":"2024-03-04T12:11:59.920921+0000","last_became_active":"2024-03-04T12:11:58.943297+0000","last_became_peered":"2024-03-04T12:11:58.943297+0000","last_unstale":"2024-03-04T12:11:59.920921+0000","last_undegraded":"2024-03-04T12:11:59.920921+0000","last_fullsized":"2024-03-04T12:11:59.920921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:56.606927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770672+0000","last_change":"2024-03-04T12:11:54.900415+0000","last_active":"2024-03-04T12:12:00.770672+0000","last_peered":"2024-03-04T12:12:00.770672+0000","last_clean":"2024-03-04T12:12:00.770672+0000","last_became_active":"2024-03-04T12:11:54.900101+0000","last_became_peered":"2024-03-04T12:11:54.900101+0000","last_unstale":"2024-03-04T12:12:00.770672+0000","last_undegraded":"2024-03-04T12:12:00.770672+0000","last_fullsized":"2024-03-04T12:12:00.770672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771029+0000","last_change":"2024-03-04T12:11:53.888493+0000","last_active":"2024-03-04T12:12:00.771029+0000","last_peered":"2024-03-04T12:12:00.771029+0000","last_clean":"2024-03-04T12:12:00.771029+0000","last_became_active":"2024-03-04T12:11:53.888223+0000","last_became_peered":"2024-03-04T12:11:53.888223+0000","last_unstale":"2024-03-04T12:12:00.771029+0000","last_undegraded":"2024-03-04T12:12:00.771029+0000","last_fullsized":"2024-03-04T12:12:00.771029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:40.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772694+0000","last_change":"2024-03-04T12:11:58.934500+0000","last_active":"2024-03-04T12:12:00.772694+0000","last_peered":"2024-03-04T12:12:00.772694+0000","last_clean":"2024-03-04T12:12:00.772694+0000","last_became_active":"2024-03-04T12:11:58.933910+0000","last_became_peered":"2024-03-04T12:11:58.933910+0000","last_unstale":"2024-03-04T12:12:00.772694+0000","last_undegraded":"2024-03-04T12:12:00.772694+0000","last_fullsized":"2024-03-04T12:12:00.772694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771718+0000","last_change":"2024-03-04T12:11:54.906835+0000","last_active":"2024-03-04T12:12:00.771718+0000","last_peered":"2024-03-04T12:12:00.771718+0000","last_clean":"2024-03-04T12:12:00.771718+0000","last_became_active":"2024-03-04T12:11:54.904477+0000","last_became_peered":"2024-03-04T12:11:54.904477+0000","last_unstale":"2024-03-04T12:12:00.771718+0000","last_undegraded":"2024-03-04T12:12:00.771718+0000","last_fullsized":"2024-03-04T12:12:00.771718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:04.123314+0000","last_change":"2024-03-04T12:11:53.889752+0000","last_active":"2024-03-04T12:12:04.123314+0000","last_peered":"2024-03-04T12:12:04.123314+0000","last_clean":"2024-03-04T12:12:04.123314+0000","last_became_active":"2024-03-04T12:11:53.889469+0000","last_became_peered":"2024-03-04T12:11:53.889469+0000","last_unstale":"2024-03-04T12:12:04.123314+0000","last_undegraded":"2024-03-04T12:12:04.123314+0000","last_fullsized":"2024-03-04T12:12:04.123314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771293+0000","last_change":"2024-03-04T12:11:58.941223+0000","last_active":"2024-03-04T12:12:00.771293+0000","last_peered":"2024-03-04T12:12:00.771293+0000","last_clean":"2024-03-04T12:12:00.771293+0000","last_became_active":"2024-03-04T12:11:58.940845+0000","last_became_peered":"2024-03-04T12:11:58.940845+0000","last_unstale":"2024-03-04T12:12:00.771293+0000","last_undegraded":"2024-03-04T12:12:00.771293+0000","last_fullsized":"2024-03-04T12:12:00.771293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772686+0000","last_change":"2024-03-04T12:11:54.902174+0000","last_active":"2024-03-04T12:12:00.772686+0000","last_peered":"2024-03-04T12:12:00.772686+0000","last_clean":"2024-03-04T12:12:00.772686+0000","last_became_active":"2024-03-04T12:11:54.901884+0000","last_became_peered":"2024-03-04T12:11:54.901884+0000","last_unstale":"2024-03-04T12:12:00.772686+0000","last_undegraded":"2024-03-04T12:12:00.772686+0000","last_fullsized":"2024-03-04T12:12:00.772686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771560+0000","last_change":"2024-03-04T12:11:53.892494+0000","last_active":"2024-03-04T12:12:00.771560+0000","last_peered":"2024-03-04T12:12:00.771560+0000","last_clean":"2024-03-04T12:12:00.771560+0000","last_became_active":"2024-03-04T12:11:53.892170+0000","last_became_peered":"2024-03-04T12:11:53.892170+0000","last_unstale":"2024-03-04T12:12:00.771560+0000","last_undegraded":"2024-03-04T12:12:00.771560+0000","last_fullsized":"2024-03-04T12:12:00.771560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773086+0000","last_change":"2024-03-04T12:11:58.945966+0000","last_active":"2024-03-04T12:12:00.773086+0000","last_peered":"2024-03-04T12:12:00.773086+0000","last_clean":"2024-03-04T12:12:00.773086+0000","last_became_active":"2024-03-04T12:11:58.945545+0000","last_became_peered":"2024-03-04T12:11:58.945545+0000","last_unstale":"2024-03-04T12:12:00.773086+0000","last_undegraded":"2024-03-04T12:12:00.773086+0000","last_fullsized":"2024-03-04T12:12:00.773086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771604+0000","last_change":"2024-03-04T12:11:54.900788+0000","last_active":"2024-03-04T12:12:00.771604+0000","last_peered":"2024-03-04T12:12:00.771604+0000","last_clean":"2024-03-04T12:12:00.771604+0000","last_became_active":"2024-03-04T12:11:54.900523+0000","last_became_peered":"2024-03-04T12:11:54.900523+0000","last_unstale":"2024-03-04T12:12:00.771604+0000","last_undegraded":"2024-03-04T12:12:00.771604+0000","last_fullsized":"2024-03-04T12:12:00.771604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773050+0000","last_change":"2024-03-04T12:11:53.887458+0000","last_active":"2024-03-04T12:12:00.773050+0000","last_peered":"2024-03-04T12:12:00.773050+0000","last_clean":"2024-03-04T12:12:00.773050+0000","last_became_active":"2024-03-04T12:11:53.887205+0000","last_became_peered":"2024-03-04T12:11:53.887205+0000","last_unstale":"2024-03-04T12:12:00.773050+0000","last_undegraded":"2024-03-04T12:12:00.773050+0000","last_fullsized":"2024-03-04T12:12:00.773050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920213+0000","last_change":"2024-03-04T12:11:58.944437+0000","last_active":"2024-03-04T12:11:59.920213+0000","last_peered":"2024-03-04T12:11:59.920213+0000","last_clean":"2024-03-04T12:11:59.920213+0000","last_became_active":"2024-03-04T12:11:58.944027+0000","last_became_peered":"2024-03-04T12:11:58.944027+0000","last_unstale":"2024-03-04T12:11:59.920213+0000","last_undegraded":"2024-03-04T12:11:59.920213+0000","last_fullsized":"2024-03-04T12:11:59.920213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:09:47.849893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773158+0000","last_change":"2024-03-04T12:11:54.904517+0000","last_active":"2024-03-04T12:12:00.773158+0000","last_peered":"2024-03-04T12:12:00.773158+0000","last_clean":"2024-03-04T12:12:00.773158+0000","last_became_active":"2024-03-04T12:11:54.904245+0000","last_became_peered":"2024-03-04T12:11:54.904245+0000","last_unstale":"2024-03-04T12:12:00.773158+0000","last_undegraded":"2024-03-04T12:12:00.773158+0000","last_fullsized":"2024-03-04T12:12:00.773158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921498+0000","last_change":"2024-03-04T12:11:53.888433+0000","last_active":"2024-03-04T12:11:59.921498+0000","last_peered":"2024-03-04T12:11:59.921498+0000","last_clean":"2024-03-04T12:11:59.921498+0000","last_became_active":"2024-03-04T12:11:53.888164+0000","last_became_peered":"2024-03-04T12:11:53.888164+0000","last_unstale":"2024-03-04T12:11:59.921498+0000","last_undegraded":"2024-03-04T12:11:59.921498+0000","last_fullsized":"2024-03-04T12:11:59.921498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920282+0000","last_change":"2024-03-04T12:11:58.937226+0000","last_active":"2024-03-04T12:11:59.920282+0000","last_peered":"2024-03-04T12:11:59.920282+0000","last_clean":"2024-03-04T12:11:59.920282+0000","last_became_active":"2024-03-04T12:11:58.936831+0000","last_became_peered":"2024-03-04T12:11:58.936831+0000","last_unstale":"2024-03-04T12:11:59.920282+0000","last_undegraded":"2024-03-04T12:11:59.920282+0000","last_fullsized":"2024-03-04T12:11:59.920282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:02.110563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770587+0000","last_change":"2024-03-04T12:11:58.946973+0000","last_active":"2024-03-04T12:12:00.770587+0000","last_peered":"2024-03-04T12:12:00.770587+0000","last_clean":"2024-03-04T12:12:00.770587+0000","last_became_active":"2024-03-04T12:11:58.946581+0000","last_became_peered":"2024-03-04T12:11:58.946581+0000","last_unstale":"2024-03-04T12:12:00.770587+0000","last_undegraded":"2024-03-04T12:12:00.770587+0000","last_fullsized":"2024-03-04T12:12:00.770587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770784+0000","last_change":"2024-03-04T12:11:54.900431+0000","last_active":"2024-03-04T12:12:00.770784+0000","last_peered":"2024-03-04T12:12:00.770784+0000","last_clean":"2024-03-04T12:12:00.770784+0000","last_became_active":"2024-03-04T12:11:54.900166+0000","last_became_peered":"2024-03-04T12:11:54.900166+0000","last_unstale":"2024-03-04T12:12:00.770784+0000","last_undegraded":"2024-03-04T12:12:00.770784+0000","last_fullsized":"2024-03-04T12:12:00.770784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:53.879757+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:53.878758+0000","last_became_peered":"2024-03-04T12:11:53.878758+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921468+0000","last_change":"2024-03-04T12:11:58.945414+0000","last_active":"2024-03-04T12:11:59.921468+0000","last_peered":"2024-03-04T12:11:59.921468+0000","last_clean":"2024-03-04T12:11:59.921468+0000","last_became_active":"2024-03-04T12:11:58.945031+0000","last_became_peered":"2024-03-04T12:11:58.945031+0000","last_unstale":"2024-03-04T12:11:59.921468+0000","last_undegraded":"2024-03-04T12:11:59.921468+0000","last_fullsized":"2024-03-04T12:11:59.921468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920217+0000","last_change":"2024-03-04T12:11:54.896991+0000","last_active":"2024-03-04T12:11:59.920217+0000","last_peered":"2024-03-04T12:11:59.920217+0000","last_clean":"2024-03-04T12:11:59.920217+0000","last_became_active":"2024-03-04T12:11:54.896725+0000","last_became_peered":"2024-03-04T12:11:54.896725+0000","last_unstale":"2024-03-04T12:11:59.920217+0000","last_undegraded":"2024-03-04T12:11:59.920217+0000","last_fullsized":"2024-03-04T12:11:59.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920530+0000","last_change":"2024-03-04T12:11:53.884328+0000","last_active":"2024-03-04T12:11:59.920530+0000","last_peered":"2024-03-04T12:11:59.920530+0000","last_clean":"2024-03-04T12:11:59.920530+0000","last_became_active":"2024-03-04T12:11:53.882865+0000","last_became_peered":"2024-03-04T12:11:53.882865+0000","last_unstale":"2024-03-04T12:11:59.920530+0000","last_undegraded":"2024-03-04T12:11:59.920530+0000","last_fullsized":"2024-03-04T12:11:59.920530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921283+0000","last_change":"2024-03-04T12:11:53.881932+0000","last_active":"2024-03-04T12:11:59.921283+0000","last_peered":"2024-03-04T12:11:59.921283+0000","last_clean":"2024-03-04T12:11:59.921283+0000","last_became_active":"2024-03-04T12:11:53.880843+0000","last_became_peered":"2024-03-04T12:11:53.880843+0000","last_unstale":"2024-03-04T12:11:59.921283+0000","last_undegraded":"2024-03-04T12:11:59.921283+0000","last_fullsized":"2024-03-04T12:11:59.921283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770941+0000","last_change":"2024-03-04T12:11:54.892168+0000","last_active":"2024-03-04T12:12:00.770941+0000","last_peered":"2024-03-04T12:12:00.770941+0000","last_clean":"2024-03-04T12:12:00.770941+0000","last_became_active":"2024-03-04T12:11:54.891886+0000","last_became_peered":"2024-03-04T12:11:54.891886+0000","last_unstale":"2024-03-04T12:12:00.770941+0000","last_undegraded":"2024-03-04T12:12:00.770941+0000","last_fullsized":"2024-03-04T12:12:00.770941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920785+0000","last_change":"2024-03-04T12:11:58.946859+0000","last_active":"2024-03-04T12:11:59.920785+0000","last_peered":"2024-03-04T12:11:59.920785+0000","last_clean":"2024-03-04T12:11:59.920785+0000","last_became_active":"2024-03-04T12:11:58.946495+0000","last_became_peered":"2024-03-04T12:11:58.946495+0000","last_unstale":"2024-03-04T12:11:59.920785+0000","last_undegraded":"2024-03-04T12:11:59.920785+0000","last_fullsized":"2024-03-04T12:11:59.920785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:25.320885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920580+0000","last_change":"2024-03-04T12:11:53.889151+0000","last_active":"2024-03-04T12:11:59.920580+0000","last_peered":"2024-03-04T12:11:59.920580+0000","last_clean":"2024-03-04T12:11:59.920580+0000","last_became_active":"2024-03-04T12:11:53.888882+0000","last_became_peered":"2024-03-04T12:11:53.888882+0000","last_unstale":"2024-03-04T12:11:59.920580+0000","last_undegraded":"2024-03-04T12:11:59.920580+0000","last_fullsized":"2024-03-04T12:11:59.920580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772788+0000","last_change":"2024-03-04T12:11:54.901800+0000","last_active":"2024-03-04T12:12:00.772788+0000","last_peered":"2024-03-04T12:12:00.772788+0000","last_clean":"2024-03-04T12:12:00.772788+0000","last_became_active":"2024-03-04T12:11:54.901502+0000","last_became_peered":"2024-03-04T12:11:54.901502+0000","last_unstale":"2024-03-04T12:12:00.772788+0000","last_undegraded":"2024-03-04T12:12:00.772788+0000","last_fullsized":"2024-03-04T12:12:00.772788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:36.238592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921109+0000","last_change":"2024-03-04T12:11:58.943607+0000","last_active":"2024-03-04T12:11:59.921109+0000","last_peered":"2024-03-04T12:11:59.921109+0000","last_clean":"2024-03-04T12:11:59.921109+0000","last_became_active":"2024-03-04T12:11:58.942837+0000","last_became_peered":"2024-03-04T12:11:58.942837+0000","last_unstale":"2024-03-04T12:11:59.921109+0000","last_undegraded":"2024-03-04T12:11:59.921109+0000","last_fullsized":"2024-03-04T12:11:59.921109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:59.670150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920801+0000","last_change":"2024-03-04T12:11:54.908034+0000","last_active":"2024-03-04T12:11:59.920801+0000","last_peered":"2024-03-04T12:11:59.920801+0000","last_clean":"2024-03-04T12:11:59.920801+0000","last_became_active":"2024-03-04T12:11:54.907761+0000","last_became_peered":"2024-03-04T12:11:54.907761+0000","last_unstale":"2024-03-04T12:11:59.920801+0000","last_undegraded":"2024-03-04T12:11:59.920801+0000","last_fullsized":"2024-03-04T12:11:59.920801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921098+0000","last_change":"2024-03-04T12:11:54.904118+0000","last_active":"2024-03-04T12:11:59.921098+0000","last_peered":"2024-03-04T12:11:59.921098+0000","last_clean":"2024-03-04T12:11:59.921098+0000","last_became_active":"2024-03-04T12:11:54.903747+0000","last_became_peered":"2024-03-04T12:11:54.903747+0000","last_unstale":"2024-03-04T12:11:59.921098+0000","last_undegraded":"2024-03-04T12:11:59.921098+0000","last_fullsized":"2024-03-04T12:11:59.921098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771638+0000","last_change":"2024-03-04T12:11:54.900197+0000","last_active":"2024-03-04T12:12:00.771638+0000","last_peered":"2024-03-04T12:12:00.771638+0000","last_clean":"2024-03-04T12:12:00.771638+0000","last_became_active":"2024-03-04T12:11:54.899939+0000","last_became_peered":"2024-03-04T12:11:54.899939+0000","last_unstale":"2024-03-04T12:12:00.771638+0000","last_undegraded":"2024-03-04T12:12:00.771638+0000","last_fullsized":"2024-03-04T12:12:00.771638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770848+0000","last_change":"2024-03-04T12:11:54.896614+0000","last_active":"2024-03-04T12:12:00.770848+0000","last_peered":"2024-03-04T12:12:00.770848+0000","last_clean":"2024-03-04T12:12:00.770848+0000","last_became_active":"2024-03-04T12:11:54.896312+0000","last_became_peered":"2024-03-04T12:11:54.896312+0000","last_unstale":"2024-03-04T12:12:00.770848+0000","last_undegraded":"2024-03-04T12:12:00.770848+0000","last_fullsized":"2024-03-04T12:12:00.770848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920990+0000","last_change":"2024-03-04T12:11:54.898597+0000","last_active":"2024-03-04T12:11:59.920990+0000","last_peered":"2024-03-04T12:11:59.920990+0000","last_clean":"2024-03-04T12:11:59.920990+0000","last_became_active":"2024-03-04T12:11:54.898339+0000","last_became_peered":"2024-03-04T12:11:54.898339+0000","last_unstale":"2024-03-04T12:11:59.920990+0000","last_undegraded":"2024-03-04T12:11:59.920990+0000","last_fullsized":"2024-03-04T12:11:59.920990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771385+0000","last_change":"2024-03-04T12:11:54.902225+0000","last_active":"2024-03-04T12:12:00.771385+0000","last_peered":"2024-03-04T12:12:00.771385+0000","last_clean":"2024-03-04T12:12:00.771385+0000","last_became_active":"2024-03-04T12:11:54.901968+0000","last_became_peered":"2024-03-04T12:11:54.901968+0000","last_unstale":"2024-03-04T12:12:00.771385+0000","last_undegraded":"2024-03-04T12:12:00.771385+0000","last_fullsized":"2024-03-04T12:12:00.771385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921621+0000","last_change":"2024-03-04T12:11:54.911542+0000","last_active":"2024-03-04T12:11:59.921621+0000","last_peered":"2024-03-04T12:11:59.921621+0000","last_clean":"2024-03-04T12:11:59.921621+0000","last_became_active":"2024-03-04T12:11:54.911032+0000","last_became_peered":"2024-03-04T12:11:54.911032+0000","last_unstale":"2024-03-04T12:11:59.921621+0000","last_undegraded":"2024-03-04T12:11:59.921621+0000","last_fullsized":"2024-03-04T12:11:59.921621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921271+0000","last_change":"2024-03-04T12:11:54.897959+0000","last_active":"2024-03-04T12:11:59.921271+0000","last_peered":"2024-03-04T12:11:59.921271+0000","last_clean":"2024-03-04T12:11:59.921271+0000","last_became_active":"2024-03-04T12:11:54.897665+0000","last_became_peered":"2024-03-04T12:11:54.897665+0000","last_unstale":"2024-03-04T12:11:59.921271+0000","last_undegraded":"2024-03-04T12:11:59.921271+0000","last_fullsized":"2024-03-04T12:11:59.921271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:54.899561+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:54.899289+0000","last_became_peered":"2024-03-04T12:11:54.899289+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920409+0000","last_change":"2024-03-04T12:11:54.897860+0000","last_active":"2024-03-04T12:11:59.920409+0000","last_peered":"2024-03-04T12:11:59.920409+0000","last_clean":"2024-03-04T12:11:59.920409+0000","last_became_active":"2024-03-04T12:11:54.897585+0000","last_became_peered":"2024-03-04T12:11:54.897585+0000","last_unstale":"2024-03-04T12:11:59.920409+0000","last_undegraded":"2024-03-04T12:11:59.920409+0000","last_fullsized":"2024-03-04T12:11:59.920409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771542+0000","last_change":"2024-03-04T12:11:54.904095+0000","last_active":"2024-03-04T12:12:00.771542+0000","last_peered":"2024-03-04T12:12:00.771542+0000","last_clean":"2024-03-04T12:12:00.771542+0000","last_became_active":"2024-03-04T12:11:54.903527+0000","last_became_peered":"2024-03-04T12:11:54.903527+0000","last_unstale":"2024-03-04T12:12:00.771542+0000","last_undegraded":"2024-03-04T12:12:00.771542+0000","last_fullsized":"2024-03-04T12:12:00.771542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770740+0000","last_change":"2024-03-04T12:11:54.904758+0000","last_active":"2024-03-04T12:12:00.770740+0000","last_peered":"2024-03-04T12:12:00.770740+0000","last_clean":"2024-03-04T12:12:00.770740+0000","last_became_active":"2024-03-04T12:11:54.904480+0000","last_became_peered":"2024-03-04T12:11:54.904480+0000","last_unstale":"2024-03-04T12:12:00.770740+0000","last_undegraded":"2024-03-04T12:12:00.770740+0000","last_fullsized":"2024-03-04T12:12:00.770740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921331+0000","last_change":"2024-03-04T12:11:54.901144+0000","last_active":"2024-03-04T12:11:59.921331+0000","last_peered":"2024-03-04T12:11:59.921331+0000","last_clean":"2024-03-04T12:11:59.921331+0000","last_became_active":"2024-03-04T12:11:54.900739+0000","last_became_peered":"2024-03-04T12:11:54.900739+0000","last_unstale":"2024-03-04T12:11:59.921331+0000","last_undegraded":"2024-03-04T12:11:59.921331+0000","last_fullsized":"2024-03-04T12:11:59.921331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771914+0000","last_change":"2024-03-04T12:11:54.905995+0000","last_active":"2024-03-04T12:12:00.771914+0000","last_peered":"2024-03-04T12:12:00.771914+0000","last_clean":"2024-03-04T12:12:00.771914+0000","last_became_active":"2024-03-04T12:11:54.904878+0000","last_became_peered":"2024-03-04T12:11:54.904878+0000","last_unstale":"2024-03-04T12:12:00.771914+0000","last_undegraded":"2024-03-04T12:12:00.771914+0000","last_fullsized":"2024-03-04T12:12:00.771914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921040+0000","last_change":"2024-03-04T12:11:54.904652+0000","last_active":"2024-03-04T12:11:59.921040+0000","last_peered":"2024-03-04T12:11:59.921040+0000","last_clean":"2024-03-04T12:11:59.921040+0000","last_became_active":"2024-03-04T12:11:54.904348+0000","last_became_peered":"2024-03-04T12:11:54.904348+0000","last_unstale":"2024-03-04T12:11:59.921040+0000","last_undegraded":"2024-03-04T12:11:59.921040+0000","last_fullsized":"2024-03-04T12:11:59.921040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:05.687532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772261+0000","last_change":"2024-03-04T12:11:54.905800+0000","last_active":"2024-03-04T12:12:00.772261+0000","last_peered":"2024-03-04T12:12:00.772261+0000","last_clean":"2024-03-04T12:12:00.772261+0000","last_became_active":"2024-03-04T12:11:54.905550+0000","last_became_peered":"2024-03-04T12:11:54.905550+0000","last_unstale":"2024-03-04T12:12:00.772261+0000","last_undegraded":"2024-03-04T12:12:00.772261+0000","last_fullsized":"2024-03-04T12:12:00.772261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771115+0000","last_change":"2024-03-04T12:11:54.909971+0000","last_active":"2024-03-04T12:12:00.771115+0000","last_peered":"2024-03-04T12:12:00.771115+0000","last_clean":"2024-03-04T12:12:00.771115+0000","last_became_active":"2024-03-04T12:11:54.909694+0000","last_became_peered":"2024-03-04T12:11:54.909694+0000","last_unstale":"2024-03-04T12:12:00.771115+0000","last_undegraded":"2024-03-04T12:12:00.771115+0000","last_fullsized":"2024-03-04T12:12:00.771115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770786+0000","last_change":"2024-03-04T12:11:54.901482+0000","last_active":"2024-03-04T12:12:00.770786+0000","last_peered":"2024-03-04T12:12:00.770786+0000","last_clean":"2024-03-04T12:12:00.770786+0000","last_became_active":"2024-03-04T12:11:54.901185+0000","last_became_peered":"2024-03-04T12:11:54.901185+0000","last_unstale":"2024-03-04T12:12:00.770786+0000","last_undegraded":"2024-03-04T12:12:00.770786+0000","last_fullsized":"2024-03-04T12:12:00.770786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770638+0000","last_change":"2024-03-04T12:11:54.902348+0000","last_active":"2024-03-04T12:12:00.770638+0000","last_peered":"2024-03-04T12:12:00.770638+0000","last_clean":"2024-03-04T12:12:00.770638+0000","last_became_active":"2024-03-04T12:11:54.902087+0000","last_became_peered":"2024-03-04T12:11:54.902087+0000","last_unstale":"2024-03-04T12:12:00.770638+0000","last_undegraded":"2024-03-04T12:12:00.770638+0000","last_fullsized":"2024-03-04T12:12:00.770638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:04:00.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920436+0000","last_change":"2024-03-04T12:11:54.904336+0000","last_active":"2024-03-04T12:11:59.920436+0000","last_peered":"2024-03-04T12:11:59.920436+0000","last_clean":"2024-03-04T12:11:59.920436+0000","last_became_active":"2024-03-04T12:11:54.904049+0000","last_became_peered":"2024-03-04T12:11:54.904049+0000","last_unstale":"2024-03-04T12:11:59.920436+0000","last_undegraded":"2024-03-04T12:11:59.920436+0000","last_fullsized":"2024-03-04T12:11:59.920436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772596+0000","last_change":"2024-03-04T12:11:54.907818+0000","last_active":"2024-03-04T12:12:00.772596+0000","last_peered":"2024-03-04T12:12:00.772596+0000","last_clean":"2024-03-04T12:12:00.772596+0000","last_became_active":"2024-03-04T12:11:54.907443+0000","last_became_peered":"2024-03-04T12:11:54.907443+0000","last_unstale":"2024-03-04T12:12:00.772596+0000","last_undegraded":"2024-03-04T12:12:00.772596+0000","last_fullsized":"2024-03-04T12:12:00.772596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:26.417645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921105+0000","last_change":"2024-03-04T12:11:54.906714+0000","last_active":"2024-03-04T12:11:59.921105+0000","last_peered":"2024-03-04T12:11:59.921105+0000","last_clean":"2024-03-04T12:11:59.921105+0000","last_became_active":"2024-03-04T12:11:54.906367+0000","last_became_peered":"2024-03-04T12:11:54.906367+0000","last_unstale":"2024-03-04T12:11:59.921105+0000","last_undegraded":"2024-03-04T12:11:59.921105+0000","last_fullsized":"2024-03-04T12:11:59.921105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921624+0000","last_change":"2024-03-04T12:11:54.908667+0000","last_active":"2024-03-04T12:11:59.921624+0000","last_peered":"2024-03-04T12:11:59.921624+0000","last_clean":"2024-03-04T12:11:59.921624+0000","last_became_active":"2024-03-04T12:11:54.908377+0000","last_became_peered":"2024-03-04T12:11:54.908377+0000","last_unstale":"2024-03-04T12:11:59.921624+0000","last_undegraded":"2024-03-04T12:11:59.921624+0000","last_fullsized":"2024-03-04T12:11:59.921624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772398+0000","last_change":"2024-03-04T12:11:54.907669+0000","last_active":"2024-03-04T12:12:00.772398+0000","last_peered":"2024-03-04T12:12:00.772398+0000","last_clean":"2024-03-04T12:12:00.772398+0000","last_became_active":"2024-03-04T12:11:54.905673+0000","last_became_peered":"2024-03-04T12:11:54.905673+0000","last_unstale":"2024-03-04T12:12:00.772398+0000","last_undegraded":"2024-03-04T12:12:00.772398+0000","last_fullsized":"2024-03-04T12:12:00.772398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:43.980065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920103+0000","last_change":"2024-03-04T12:11:54.904733+0000","last_active":"2024-03-04T12:11:59.920103+0000","last_peered":"2024-03-04T12:11:59.920103+0000","last_clean":"2024-03-04T12:11:59.920103+0000","last_became_active":"2024-03-04T12:11:54.903680+0000","last_became_peered":"2024-03-04T12:11:54.903680+0000","last_unstale":"2024-03-04T12:11:59.920103+0000","last_undegraded":"2024-03-04T12:11:59.920103+0000","last_fullsized":"2024-03-04T12:11:59.920103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771078+0000","last_change":"2024-03-04T12:11:54.902367+0000","last_active":"2024-03-04T12:12:00.771078+0000","last_peered":"2024-03-04T12:12:00.771078+0000","last_clean":"2024-03-04T12:12:00.771078+0000","last_became_active":"2024-03-04T12:11:54.902075+0000","last_became_peered":"2024-03-04T12:11:54.902075+0000","last_unstale":"2024-03-04T12:12:00.771078+0000","last_undegraded":"2024-03-04T12:12:00.771078+0000","last_fullsized":"2024-03-04T12:12:00.771078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771398+0000","last_change":"2024-03-04T12:11:54.899399+0000","last_active":"2024-03-04T12:12:00.771398+0000","last_peered":"2024-03-04T12:12:00.771398+0000","last_clean":"2024-03-04T12:12:00.771398+0000","last_became_active":"2024-03-04T12:11:54.899084+0000","last_became_peered":"2024-03-04T12:11:54.899084+0000","last_unstale":"2024-03-04T12:12:00.771398+0000","last_undegraded":"2024-03-04T12:12:00.771398+0000","last_fullsized":"2024-03-04T12:12:00.771398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772929+0000","last_change":"2024-03-04T12:11:54.903214+0000","last_active":"2024-03-04T12:12:00.772929+0000","last_peered":"2024-03-04T12:12:00.772929+0000","last_clean":"2024-03-04T12:12:00.772929+0000","last_became_active":"2024-03-04T12:11:54.901705+0000","last_became_peered":"2024-03-04T12:11:54.901705+0000","last_unstale":"2024-03-04T12:12:00.772929+0000","last_undegraded":"2024-03-04T12:12:00.772929+0000","last_fullsized":"2024-03-04T12:12:00.772929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771893+0000","last_change":"2024-03-04T12:11:54.910691+0000","last_active":"2024-03-04T12:12:00.771893+0000","last_peered":"2024-03-04T12:12:00.771893+0000","last_clean":"2024-03-04T12:12:00.771893+0000","last_became_active":"2024-03-04T12:11:54.910198+0000","last_became_peered":"2024-03-04T12:11:54.910198+0000","last_unstale":"2024-03-04T12:12:00.771893+0000","last_undegraded":"2024-03-04T12:12:00.771893+0000","last_fullsized":"2024-03-04T12:12:00.771893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771263+0000","last_change":"2024-03-04T12:11:54.898528+0000","last_active":"2024-03-04T12:12:00.771263+0000","last_peered":"2024-03-04T12:12:00.771263+0000","last_clean":"2024-03-04T12:12:00.771263+0000","last_became_active":"2024-03-04T12:11:54.897521+0000","last_became_peered":"2024-03-04T12:11:54.897521+0000","last_unstale":"2024-03-04T12:12:00.771263+0000","last_undegraded":"2024-03-04T12:12:00.771263+0000","last_fullsized":"2024-03-04T12:12:00.771263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771266+0000","last_change":"2024-03-04T12:11:54.901718+0000","last_active":"2024-03-04T12:12:00.771266+0000","last_peered":"2024-03-04T12:12:00.771266+0000","last_clean":"2024-03-04T12:12:00.771266+0000","last_became_active":"2024-03-04T12:11:54.901395+0000","last_became_peered":"2024-03-04T12:11:54.901395+0000","last_unstale":"2024-03-04T12:12:00.771266+0000","last_undegraded":"2024-03-04T12:12:00.771266+0000","last_fullsized":"2024-03-04T12:12:00.771266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772905+0000","last_change":"2024-03-04T12:11:54.907257+0000","last_active":"2024-03-04T12:12:00.772905+0000","last_peered":"2024-03-04T12:12:00.772905+0000","last_clean":"2024-03-04T12:12:00.772905+0000","last_became_active":"2024-03-04T12:11:54.906648+0000","last_became_peered":"2024-03-04T12:11:54.906648+0000","last_unstale":"2024-03-04T12:12:00.772905+0000","last_undegraded":"2024-03-04T12:12:00.772905+0000","last_fullsized":"2024-03-04T12:12:00.772905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:02.739456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:12:22.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:12:22.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.379+0000 7f8b52b9d640 1 Processor -- start 2024-03-04T12:12:22.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.379+0000 7f8b52b9d640 1 -- start start 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b52b9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b52b9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b52b9d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c15df50 con 0x7f8b4c15db70 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b52b9d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d2310 con 0x7f8b4c0d0740 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b52b9d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d25f0 con 0x7f8b4c0b6900 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b5139a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b5139a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40216/0 (socket says 172.21.15.148:40216) 2024-03-04T12:12:22.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b51b9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b4c15db70 0x7f8b4c0d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47460/0 (socket says 172.21.15.148:47460) 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.380+0000 7f8b5139a640 1 -- 172.21.15.148:0/3832930320 learned_addr learned my addr 172.21.15.148:0/3832930320 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5239c640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5139a640 1 -- 172.21.15.148:0/3832930320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 msgr2=0x7f8b4c0b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5139a640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c0b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5139a640 1 -- 172.21.15.148:0/3832930320 >> v1:172.21.15.148:6789/0 conn(0x7f8b4c15db70 legacy=0x7f8b4c0d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5139a640 1 -- 172.21.15.148:0/3832930320 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0d28d0 con 0x7f8b4c0d0740 2024-03-04T12:12:22.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b50b99640 1 -- 172.21.15.148:0/3832930320 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283772487 0 0) 0x7f8b4c15df50 con 0x7f8b4c15db70 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.381+0000 7f8b5139a640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009870 tx=0x7f8b3c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e025bb681b98c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b50b99640 1 -- 172.21.15.148:0/3832930320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c034070 con 0x7f8b4c0d0740 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b50b99640 1 -- 172.21.15.148:0/3832930320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c030840 con 0x7f8b4c0d0740 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b50b99640 1 -- 172.21.15.148:0/3832930320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c030c10 con 0x7f8b4c0d0740 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3832930320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c0d0740 msgr2=0x7f8b4c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009870 tx=0x7f8b3c02b980 comp rx=0 tx=0).stop 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3832930320 shutdown_connections 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c0b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3832930320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0d0bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3832930320 >> 172.21.15.148:0/3832930320 conn(0x7f8b4c072820 msgr2=0x7f8b4c072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3832930320 shutdown_connections 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.382+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3832930320 wait complete. 2024-03-04T12:12:22.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.383+0000 7f8b52b9d640 1 Processor -- start 2024-03-04T12:12:22.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.383+0000 7f8b52b9d640 1 -- start start 2024-03-04T12:12:22.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d1ff0 con 0x7f8b4c15db70 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d2e90 con 0x7f8b4c0d0740 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b52b9d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d33b0 con 0x7f8b4c0b6900 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40232/0 (socket says 172.21.15.148:40232) 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 -- 172.21.15.148:0/3399763664 learned_addr learned my addr 172.21.15.148:0/3399763664 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5139a640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b51b9b640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 -- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 msgr2=0x7f8b4c160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 -- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b4c0d0740 msgr2=0x7f8b4c0cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.384+0000 7f8b5239c640 1 -- 172.21.15.148:0/3399763664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0c7480 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b5239c640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b44007e40 tx=0x7f8b4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014620 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44019070 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014a20 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c0c7760 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.385+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0c7d20 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.386+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c0cc3a0 con 0x7f8b4c15db70 2024-03-04T12:12:22.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.386+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8b44014bc0 con 0x7f8b4c15db70 2024-03-04T12:12:22.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.387+0000 7f8b427fc640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 0x7f8b3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.387+0000 7f8b51b9b640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 0x7f8b3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.387+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b4408acf0 con 0x7f8b4c15db70 2024-03-04T12:12:22.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.387+0000 7f8b51b9b640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 0x7f8b3006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007fd0 tx=0x7f8b48007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.391+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b4408f070 con 0x7f8b4c15db70 2024-03-04T12:12:22.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.558+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b4c04f100 con 0x7f8b3006b4d0 2024-03-04T12:12:22.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.565+0000 7f8b427fc640 1 -- 172.21.15.148:0/3399763664 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f8b4c04f100 con 0x7f8b3006b4d0 2024-03-04T12:12:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:22.572 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:12:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 msgr2=0x7f8b3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 0x7f8b3006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007fd0 tx=0x7f8b48007de0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 msgr2=0x7f8b4c0cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b44007e40 tx=0x7f8b4400e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 shutdown_connections 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8b3006b4d0 0x7f8b3006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4c15db70 0x7f8b4c0cbe60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b4c0d0740 0x7f8b4c0cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 --2- 172.21.15.148:0/3399763664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b4c0b6900 0x7f8b4c160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 >> 172.21.15.148:0/3399763664 conn(0x7f8b4c072820 msgr2=0x7f8b4c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 shutdown_connections 2024-03-04T12:12:22.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.569+0000 7f8b52b9d640 1 -- 172.21.15.148:0/3399763664 wait complete. 2024-03-04T12:12:22.581 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-04T12:12:20.719571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003262"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771745+0000","last_change":"2024-03-04T12:11:53.892196+0000","last_active":"2024-03-04T12:12:00.771745+0000","last_peered":"2024-03-04T12:12:00.771745+0000","last_clean":"2024-03-04T12:12:00.771745+0000","last_became_active":"2024-03-04T12:11:53.891878+0000","last_became_peered":"2024-03-04T12:11:53.891878+0000","last_unstale":"2024-03-04T12:12:00.771745+0000","last_undegraded":"2024-03-04T12:12:00.771745+0000","last_fullsized":"2024-03-04T12:12:00.771745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:54.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921065+0000","last_change":"2024-03-04T12:11:54.905432+0000","last_active":"2024-03-04T12:11:59.921065+0000","last_peered":"2024-03-04T12:11:59.921065+0000","last_clean":"2024-03-04T12:11:59.921065+0000","last_became_active":"2024-03-04T12:11:54.905131+0000","last_became_peered":"2024-03-04T12:11:54.905131+0000","last_unstale":"2024-03-04T12:11:59.921065+0000","last_undegraded":"2024-03-04T12:11:59.921065+0000","last_fullsized":"2024-03-04T12:11:59.921065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920122+0000","last_change":"2024-03-04T12:11:58.938162+0000","last_active":"2024-03-04T12:11:59.920122+0000","last_peered":"2024-03-04T12:11:59.920122+0000","last_clean":"2024-03-04T12:11:59.920122+0000","last_became_active":"2024-03-04T12:11:58.937622+0000","last_became_peered":"2024-03-04T12:11:58.937622+0000","last_unstale":"2024-03-04T12:11:59.920122+0000","last_undegraded":"2024-03-04T12:11:59.920122+0000","last_fullsized":"2024-03-04T12:11:59.920122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:33:55.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921215+0000","last_change":"2024-03-04T12:11:53.881535+0000","last_active":"2024-03-04T12:11:59.921215+0000","last_peered":"2024-03-04T12:11:59.921215+0000","last_clean":"2024-03-04T12:11:59.921215+0000","last_became_active":"2024-03-04T12:11:53.880591+0000","last_became_peered":"2024-03-04T12:11:53.880591+0000","last_unstale":"2024-03-04T12:11:59.921215+0000","last_undegraded":"2024-03-04T12:11:59.921215+0000","last_fullsized":"2024-03-04T12:11:59.921215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771443+0000","last_change":"2024-03-04T12:11:54.906418+0000","last_active":"2024-03-04T12:12:00.771443+0000","last_peered":"2024-03-04T12:12:00.771443+0000","last_clean":"2024-03-04T12:12:00.771443+0000","last_became_active":"2024-03-04T12:11:54.906146+0000","last_became_peered":"2024-03-04T12:11:54.906146+0000","last_unstale":"2024-03-04T12:12:00.771443+0000","last_undegraded":"2024-03-04T12:12:00.771443+0000","last_fullsized":"2024-03-04T12:12:00.771443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771197+0000","last_change":"2024-03-04T12:11:58.945630+0000","last_active":"2024-03-04T12:12:00.771197+0000","last_peered":"2024-03-04T12:12:00.771197+0000","last_clean":"2024-03-04T12:12:00.771197+0000","last_became_active":"2024-03-04T12:11:58.945230+0000","last_became_peered":"2024-03-04T12:11:58.945230+0000","last_unstale":"2024-03-04T12:12:00.771197+0000","last_undegraded":"2024-03-04T12:12:00.771197+0000","last_fullsized":"2024-03-04T12:12:00.771197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:50.652152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.895431+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.895122+0000","last_became_peered":"2024-03-04T12:11:53.895122+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:54.330076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771997+0000","last_change":"2024-03-04T12:11:54.902410+0000","last_active":"2024-03-04T12:12:00.771997+0000","last_peered":"2024-03-04T12:12:00.771997+0000","last_clean":"2024-03-04T12:12:00.771997+0000","last_became_active":"2024-03-04T12:11:54.902131+0000","last_became_peered":"2024-03-04T12:11:54.902131+0000","last_unstale":"2024-03-04T12:12:00.771997+0000","last_undegraded":"2024-03-04T12:12:00.771997+0000","last_fullsized":"2024-03-04T12:12:00.771997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772908+0000","last_change":"2024-03-04T12:11:58.943323+0000","last_active":"2024-03-04T12:12:00.772908+0000","last_peered":"2024-03-04T12:12:00.772908+0000","last_clean":"2024-03-04T12:12:00.772908+0000","last_became_active":"2024-03-04T12:11:58.942878+0000","last_became_peered":"2024-03-04T12:11:58.942878+0000","last_unstale":"2024-03-04T12:12:00.772908+0000","last_undegraded":"2024-03-04T12:12:00.772908+0000","last_fullsized":"2024-03-04T12:12:00.772908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771622+0000","last_change":"2024-03-04T12:11:53.893583+0000","last_active":"2024-03-04T12:12:00.771622+0000","last_peered":"2024-03-04T12:12:00.771622+0000","last_clean":"2024-03-04T12:12:00.771622+0000","last_became_active":"2024-03-04T12:11:53.893295+0000","last_became_peered":"2024-03-04T12:11:53.893295+0000","last_unstale":"2024-03-04T12:12:00.771622+0000","last_undegraded":"2024-03-04T12:12:00.771622+0000","last_fullsized":"2024-03-04T12:12:00.771622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772748+0000","last_change":"2024-03-04T12:11:54.901339+0000","last_active":"2024-03-04T12:12:00.772748+0000","last_peered":"2024-03-04T12:12:00.772748+0000","last_clean":"2024-03-04T12:12:00.772748+0000","last_became_active":"2024-03-04T12:11:54.900914+0000","last_became_peered":"2024-03-04T12:11:54.900914+0000","last_unstale":"2024-03-04T12:12:00.772748+0000","last_undegraded":"2024-03-04T12:12:00.772748+0000","last_fullsized":"2024-03-04T12:12:00.772748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773092+0000","last_change":"2024-03-04T12:11:58.937633+0000","last_active":"2024-03-04T12:12:00.773092+0000","last_peered":"2024-03-04T12:12:00.773092+0000","last_clean":"2024-03-04T12:12:00.773092+0000","last_became_active":"2024-03-04T12:11:58.936727+0000","last_became_peered":"2024-03-04T12:11:58.936727+0000","last_unstale":"2024-03-04T12:12:00.773092+0000","last_undegraded":"2024-03-04T12:12:00.773092+0000","last_fullsized":"2024-03-04T12:12:00.773092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:03:00.255209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921104+0000","last_change":"2024-03-04T12:11:54.905529+0000","last_active":"2024-03-04T12:11:59.921104+0000","last_peered":"2024-03-04T12:11:59.921104+0000","last_clean":"2024-03-04T12:11:59.921104+0000","last_became_active":"2024-03-04T12:11:54.904590+0000","last_became_peered":"2024-03-04T12:11:54.904590+0000","last_unstale":"2024-03-04T12:11:59.921104+0000","last_undegraded":"2024-03-04T12:11:59.921104+0000","last_fullsized":"2024-03-04T12:11:59.921104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772191+0000","last_change":"2024-03-04T12:11:53.892967+0000","last_active":"2024-03-04T12:12:00.772191+0000","last_peered":"2024-03-04T12:12:00.772191+0000","last_clean":"2024-03-04T12:12:00.772191+0000","last_became_active":"2024-03-04T12:11:53.892698+0000","last_became_peered":"2024-03-04T12:11:53.892698+0000","last_unstale":"2024-03-04T12:12:00.772191+0000","last_undegraded":"2024-03-04T12:12:00.772191+0000","last_fullsized":"2024-03-04T12:12:00.772191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772796+0000","last_change":"2024-03-04T12:11:58.947133+0000","last_active":"2024-03-04T12:12:00.772796+0000","last_peered":"2024-03-04T12:12:00.772796+0000","last_clean":"2024-03-04T12:12:00.772796+0000","last_became_active":"2024-03-04T12:11:58.946617+0000","last_became_peered":"2024-03-04T12:11:58.946617+0000","last_unstale":"2024-03-04T12:12:00.772796+0000","last_undegraded":"2024-03-04T12:12:00.772796+0000","last_fullsized":"2024-03-04T12:12:00.772796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920487+0000","last_change":"2024-03-04T12:11:54.896623+0000","last_active":"2024-03-04T12:11:59.920487+0000","last_peered":"2024-03-04T12:11:59.920487+0000","last_clean":"2024-03-04T12:11:59.920487+0000","last_became_active":"2024-03-04T12:11:54.896359+0000","last_became_peered":"2024-03-04T12:11:54.896359+0000","last_unstale":"2024-03-04T12:11:59.920487+0000","last_undegraded":"2024-03-04T12:11:59.920487+0000","last_fullsized":"2024-03-04T12:11:59.920487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771403+0000","last_change":"2024-03-04T12:11:53.886825+0000","last_active":"2024-03-04T12:12:00.771403+0000","last_peered":"2024-03-04T12:12:00.771403+0000","last_clean":"2024-03-04T12:12:00.771403+0000","last_became_active":"2024-03-04T12:11:53.886525+0000","last_became_peered":"2024-03-04T12:11:53.886525+0000","last_unstale":"2024-03-04T12:12:00.771403+0000","last_undegraded":"2024-03-04T12:12:00.771403+0000","last_fullsized":"2024-03-04T12:12:00.771403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772607+0000","last_change":"2024-03-04T12:11:58.946327+0000","last_active":"2024-03-04T12:12:00.772607+0000","last_peered":"2024-03-04T12:12:00.772607+0000","last_clean":"2024-03-04T12:12:00.772607+0000","last_became_active":"2024-03-04T12:11:58.945962+0000","last_became_peered":"2024-03-04T12:11:58.945962+0000","last_unstale":"2024-03-04T12:12:00.772607+0000","last_undegraded":"2024-03-04T12:12:00.772607+0000","last_fullsized":"2024-03-04T12:12:00.772607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920261+0000","last_change":"2024-03-04T12:11:54.903139+0000","last_active":"2024-03-04T12:11:59.920261+0000","last_peered":"2024-03-04T12:11:59.920261+0000","last_clean":"2024-03-04T12:11:59.920261+0000","last_became_active":"2024-03-04T12:11:54.902809+0000","last_became_peered":"2024-03-04T12:11:54.902809+0000","last_unstale":"2024-03-04T12:11:59.920261+0000","last_undegraded":"2024-03-04T12:11:59.920261+0000","last_fullsized":"2024-03-04T12:11:59.920261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770887+0000","last_change":"2024-03-04T12:11:53.889802+0000","last_active":"2024-03-04T12:12:00.770887+0000","last_peered":"2024-03-04T12:12:00.770887+0000","last_clean":"2024-03-04T12:12:00.770887+0000","last_became_active":"2024-03-04T12:11:53.889550+0000","last_became_peered":"2024-03-04T12:11:53.889550+0000","last_unstale":"2024-03-04T12:12:00.770887+0000","last_undegraded":"2024-03-04T12:12:00.770887+0000","last_fullsized":"2024-03-04T12:12:00.770887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771267+0000","last_change":"2024-03-04T12:11:58.941310+0000","last_active":"2024-03-04T12:12:00.771267+0000","last_peered":"2024-03-04T12:12:00.771267+0000","last_clean":"2024-03-04T12:12:00.771267+0000","last_became_active":"2024-03-04T12:11:58.940932+0000","last_became_peered":"2024-03-04T12:11:58.940932+0000","last_unstale":"2024-03-04T12:12:00.771267+0000","last_undegraded":"2024-03-04T12:12:00.771267+0000","last_fullsized":"2024-03-04T12:12:00.771267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772213+0000","last_change":"2024-03-04T12:11:54.900654+0000","last_active":"2024-03-04T12:12:00.772213+0000","last_peered":"2024-03-04T12:12:00.772213+0000","last_clean":"2024-03-04T12:12:00.772213+0000","last_became_active":"2024-03-04T12:11:54.900311+0000","last_became_peered":"2024-03-04T12:11:54.900311+0000","last_unstale":"2024-03-04T12:12:00.772213+0000","last_undegraded":"2024-03-04T12:12:00.772213+0000","last_fullsized":"2024-03-04T12:12:00.772213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:23.687708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771071+0000","last_change":"2024-03-04T12:11:53.892220+0000","last_active":"2024-03-04T12:12:00.771071+0000","last_peered":"2024-03-04T12:12:00.771071+0000","last_clean":"2024-03-04T12:12:00.771071+0000","last_became_active":"2024-03-04T12:11:53.891859+0000","last_became_peered":"2024-03-04T12:11:53.891859+0000","last_unstale":"2024-03-04T12:12:00.771071+0000","last_undegraded":"2024-03-04T12:12:00.771071+0000","last_fullsized":"2024-03-04T12:12:00.771071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771968+0000","last_change":"2024-03-04T12:11:58.947661+0000","last_active":"2024-03-04T12:12:00.771968+0000","last_peered":"2024-03-04T12:12:00.771968+0000","last_clean":"2024-03-04T12:12:00.771968+0000","last_became_active":"2024-03-04T12:11:58.946889+0000","last_became_peered":"2024-03-04T12:11:58.946889+0000","last_unstale":"2024-03-04T12:12:00.771968+0000","last_undegraded":"2024-03-04T12:12:00.771968+0000","last_fullsized":"2024-03-04T12:12:00.771968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:48.995386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771351+0000","last_change":"2024-03-04T12:11:54.898183+0000","last_active":"2024-03-04T12:12:00.771351+0000","last_peered":"2024-03-04T12:12:00.771351+0000","last_clean":"2024-03-04T12:12:00.771351+0000","last_became_active":"2024-03-04T12:11:54.897873+0000","last_became_peered":"2024-03-04T12:11:54.897873+0000","last_unstale":"2024-03-04T12:12:00.771351+0000","last_undegraded":"2024-03-04T12:12:00.771351+0000","last_fullsized":"2024-03-04T12:12:00.771351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921618+0000","last_change":"2024-03-04T12:11:53.893093+0000","last_active":"2024-03-04T12:11:59.921618+0000","last_peered":"2024-03-04T12:11:59.921618+0000","last_clean":"2024-03-04T12:11:59.921618+0000","last_became_active":"2024-03-04T12:11:53.892821+0000","last_became_peered":"2024-03-04T12:11:53.892821+0000","last_unstale":"2024-03-04T12:11:59.921618+0000","last_undegraded":"2024-03-04T12:11:59.921618+0000","last_fullsized":"2024-03-04T12:11:59.921618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771969+0000","last_change":"2024-03-04T12:11:58.936275+0000","last_active":"2024-03-04T12:12:00.771969+0000","last_peered":"2024-03-04T12:12:00.771969+0000","last_clean":"2024-03-04T12:12:00.771969+0000","last_became_active":"2024-03-04T12:11:58.935879+0000","last_became_peered":"2024-03-04T12:11:58.935879+0000","last_unstale":"2024-03-04T12:12:00.771969+0000","last_undegraded":"2024-03-04T12:12:00.771969+0000","last_fullsized":"2024-03-04T12:12:00.771969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:57.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921287+0000","last_change":"2024-03-04T12:11:54.905996+0000","last_active":"2024-03-04T12:11:59.921287+0000","last_peered":"2024-03-04T12:11:59.921287+0000","last_clean":"2024-03-04T12:11:59.921287+0000","last_became_active":"2024-03-04T12:11:54.905164+0000","last_became_peered":"2024-03-04T12:11:54.905164+0000","last_unstale":"2024-03-04T12:11:59.921287+0000","last_undegraded":"2024-03-04T12:11:59.921287+0000","last_fullsized":"2024-03-04T12:11:59.921287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771364+0000","last_change":"2024-03-04T12:11:53.889023+0000","last_active":"2024-03-04T12:12:00.771364+0000","last_peered":"2024-03-04T12:12:00.771364+0000","last_clean":"2024-03-04T12:12:00.771364+0000","last_became_active":"2024-03-04T12:11:53.888732+0000","last_became_peered":"2024-03-04T12:11:53.888732+0000","last_unstale":"2024-03-04T12:12:00.771364+0000","last_undegraded":"2024-03-04T12:12:00.771364+0000","last_fullsized":"2024-03-04T12:12:00.771364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:25.412403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920041+0000","last_change":"2024-03-04T12:11:58.933844+0000","last_active":"2024-03-04T12:11:59.920041+0000","last_peered":"2024-03-04T12:11:59.920041+0000","last_clean":"2024-03-04T12:11:59.920041+0000","last_became_active":"2024-03-04T12:11:58.933456+0000","last_became_peered":"2024-03-04T12:11:58.933456+0000","last_unstale":"2024-03-04T12:11:59.920041+0000","last_undegraded":"2024-03-04T12:11:59.920041+0000","last_fullsized":"2024-03-04T12:11:59.920041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771789+0000","last_change":"2024-03-04T12:11:54.900251+0000","last_active":"2024-03-04T12:12:00.771789+0000","last_peered":"2024-03-04T12:12:00.771789+0000","last_clean":"2024-03-04T12:12:00.771789+0000","last_became_active":"2024-03-04T12:11:54.899988+0000","last_became_peered":"2024-03-04T12:11:54.899988+0000","last_unstale":"2024-03-04T12:12:00.771789+0000","last_undegraded":"2024-03-04T12:12:00.771789+0000","last_fullsized":"2024-03-04T12:12:00.771789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771966+0000","last_change":"2024-03-04T12:11:53.893557+0000","last_active":"2024-03-04T12:12:00.771966+0000","last_peered":"2024-03-04T12:12:00.771966+0000","last_clean":"2024-03-04T12:12:00.771966+0000","last_became_active":"2024-03-04T12:11:53.893170+0000","last_became_peered":"2024-03-04T12:11:53.893170+0000","last_unstale":"2024-03-04T12:12:00.771966+0000","last_undegraded":"2024-03-04T12:12:00.771966+0000","last_fullsized":"2024-03-04T12:12:00.771966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:04.682290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770797+0000","last_change":"2024-03-04T12:11:58.944095+0000","last_active":"2024-03-04T12:12:00.770797+0000","last_peered":"2024-03-04T12:12:00.770797+0000","last_clean":"2024-03-04T12:12:00.770797+0000","last_became_active":"2024-03-04T12:11:58.943691+0000","last_became_peered":"2024-03-04T12:11:58.943691+0000","last_unstale":"2024-03-04T12:12:00.770797+0000","last_undegraded":"2024-03-04T12:12:00.770797+0000","last_fullsized":"2024-03-04T12:12:00.770797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:43.931407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772840+0000","last_change":"2024-03-04T12:11:54.909256+0000","last_active":"2024-03-04T12:12:00.772840+0000","last_peered":"2024-03-04T12:12:00.772840+0000","last_clean":"2024-03-04T12:12:00.772840+0000","last_became_active":"2024-03-04T12:11:54.908971+0000","last_became_peered":"2024-03-04T12:11:54.908971+0000","last_unstale":"2024-03-04T12:12:00.772840+0000","last_undegraded":"2024-03-04T12:12:00.772840+0000","last_fullsized":"2024-03-04T12:12:00.772840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771832+0000","last_change":"2024-03-04T12:11:53.881754+0000","last_active":"2024-03-04T12:12:00.771832+0000","last_peered":"2024-03-04T12:12:00.771832+0000","last_clean":"2024-03-04T12:12:00.771832+0000","last_became_active":"2024-03-04T12:11:53.881149+0000","last_became_peered":"2024-03-04T12:11:53.881149+0000","last_unstale":"2024-03-04T12:12:00.771832+0000","last_undegraded":"2024-03-04T12:12:00.771832+0000","last_fullsized":"2024-03-04T12:12:00.771832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771018+0000","last_change":"2024-03-04T12:11:58.939152+0000","last_active":"2024-03-04T12:12:00.771018+0000","last_peered":"2024-03-04T12:12:00.771018+0000","last_clean":"2024-03-04T12:12:00.771018+0000","last_became_active":"2024-03-04T12:11:58.938820+0000","last_became_peered":"2024-03-04T12:11:58.938820+0000","last_unstale":"2024-03-04T12:12:00.771018+0000","last_undegraded":"2024-03-04T12:12:00.771018+0000","last_fullsized":"2024-03-04T12:12:00.771018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773199+0000","last_change":"2024-03-04T12:11:54.893605+0000","last_active":"2024-03-04T12:12:00.773199+0000","last_peered":"2024-03-04T12:12:00.773199+0000","last_clean":"2024-03-04T12:12:00.773199+0000","last_became_active":"2024-03-04T12:11:54.893164+0000","last_became_peered":"2024-03-04T12:11:54.893164+0000","last_unstale":"2024-03-04T12:12:00.773199+0000","last_undegraded":"2024-03-04T12:12:00.773199+0000","last_fullsized":"2024-03-04T12:12:00.773199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:32.197141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771856+0000","last_change":"2024-03-04T12:11:53.884670+0000","last_active":"2024-03-04T12:12:00.771856+0000","last_peered":"2024-03-04T12:12:00.771856+0000","last_clean":"2024-03-04T12:12:00.771856+0000","last_became_active":"2024-03-04T12:11:53.884396+0000","last_became_peered":"2024-03-04T12:11:53.884396+0000","last_unstale":"2024-03-04T12:12:00.771856+0000","last_undegraded":"2024-03-04T12:12:00.771856+0000","last_fullsized":"2024-03-04T12:12:00.771856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771791+0000","last_change":"2024-03-04T12:11:58.939922+0000","last_active":"2024-03-04T12:12:00.771791+0000","last_peered":"2024-03-04T12:12:00.771791+0000","last_clean":"2024-03-04T12:12:00.771791+0000","last_became_active":"2024-03-04T12:11:58.939562+0000","last_became_peered":"2024-03-04T12:11:58.939562+0000","last_unstale":"2024-03-04T12:12:00.771791+0000","last_undegraded":"2024-03-04T12:12:00.771791+0000","last_fullsized":"2024-03-04T12:12:00.771791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770827+0000","last_change":"2024-03-04T12:11:54.898108+0000","last_active":"2024-03-04T12:12:00.770827+0000","last_peered":"2024-03-04T12:12:00.770827+0000","last_clean":"2024-03-04T12:12:00.770827+0000","last_became_active":"2024-03-04T12:11:54.897832+0000","last_became_peered":"2024-03-04T12:11:54.897832+0000","last_unstale":"2024-03-04T12:12:00.770827+0000","last_undegraded":"2024-03-04T12:12:00.770827+0000","last_fullsized":"2024-03-04T12:12:00.770827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921186+0000","last_change":"2024-03-04T12:11:53.884846+0000","last_active":"2024-03-04T12:11:59.921186+0000","last_peered":"2024-03-04T12:11:59.921186+0000","last_clean":"2024-03-04T12:11:59.921186+0000","last_became_active":"2024-03-04T12:11:53.884567+0000","last_became_peered":"2024-03-04T12:11:53.884567+0000","last_unstale":"2024-03-04T12:11:59.921186+0000","last_undegraded":"2024-03-04T12:11:59.921186+0000","last_fullsized":"2024-03-04T12:11:59.921186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771888+0000","last_change":"2024-03-04T12:11:58.936068+0000","last_active":"2024-03-04T12:12:00.771888+0000","last_peered":"2024-03-04T12:12:00.771888+0000","last_clean":"2024-03-04T12:12:00.771888+0000","last_became_active":"2024-03-04T12:11:58.935721+0000","last_became_peered":"2024-03-04T12:11:58.935721+0000","last_unstale":"2024-03-04T12:12:00.771888+0000","last_undegraded":"2024-03-04T12:12:00.771888+0000","last_fullsized":"2024-03-04T12:12:00.771888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770381+0000","last_change":"2024-03-04T12:11:54.893726+0000","last_active":"2024-03-04T12:12:00.770381+0000","last_peered":"2024-03-04T12:12:00.770381+0000","last_clean":"2024-03-04T12:12:00.770381+0000","last_became_active":"2024-03-04T12:11:54.893038+0000","last_became_peered":"2024-03-04T12:11:54.893038+0000","last_unstale":"2024-03-04T12:12:00.770381+0000","last_undegraded":"2024-03-04T12:12:00.770381+0000","last_fullsized":"2024-03-04T12:12:00.770381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:21.280056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770859+0000","last_change":"2024-03-04T12:11:53.893383+0000","last_active":"2024-03-04T12:12:00.770859+0000","last_peered":"2024-03-04T12:12:00.770859+0000","last_clean":"2024-03-04T12:12:00.770859+0000","last_became_active":"2024-03-04T12:11:53.892630+0000","last_became_peered":"2024-03-04T12:11:53.892630+0000","last_unstale":"2024-03-04T12:12:00.770859+0000","last_undegraded":"2024-03-04T12:12:00.770859+0000","last_fullsized":"2024-03-04T12:12:00.770859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:31.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771246+0000","last_change":"2024-03-04T12:11:58.946451+0000","last_active":"2024-03-04T12:12:00.771246+0000","last_peered":"2024-03-04T12:12:00.771246+0000","last_clean":"2024-03-04T12:12:00.771246+0000","last_became_active":"2024-03-04T12:11:58.946080+0000","last_became_peered":"2024-03-04T12:11:58.946080+0000","last_unstale":"2024-03-04T12:12:00.771246+0000","last_undegraded":"2024-03-04T12:12:00.771246+0000","last_fullsized":"2024-03-04T12:12:00.771246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:06.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920426+0000","last_change":"2024-03-04T12:11:54.902080+0000","last_active":"2024-03-04T12:11:59.920426+0000","last_peered":"2024-03-04T12:11:59.920426+0000","last_clean":"2024-03-04T12:11:59.920426+0000","last_became_active":"2024-03-04T12:11:54.901382+0000","last_became_peered":"2024-03-04T12:11:54.901382+0000","last_unstale":"2024-03-04T12:11:59.920426+0000","last_undegraded":"2024-03-04T12:11:59.920426+0000","last_fullsized":"2024-03-04T12:11:59.920426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772165+0000","last_change":"2024-03-04T12:11:53.883991+0000","last_active":"2024-03-04T12:12:00.772165+0000","last_peered":"2024-03-04T12:12:00.772165+0000","last_clean":"2024-03-04T12:12:00.772165+0000","last_became_active":"2024-03-04T12:11:53.881700+0000","last_became_peered":"2024-03-04T12:11:53.881700+0000","last_unstale":"2024-03-04T12:12:00.772165+0000","last_undegraded":"2024-03-04T12:12:00.772165+0000","last_fullsized":"2024-03-04T12:12:00.772165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770976+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:12:00.770976+0000","last_peered":"2024-03-04T12:12:00.770976+0000","last_clean":"2024-03-04T12:12:00.770976+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:12:00.770976+0000","last_undegraded":"2024-03-04T12:12:00.770976+0000","last_fullsized":"2024-03-04T12:12:00.770976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921000+0000","last_change":"2024-03-04T12:11:58.939590+0000","last_active":"2024-03-04T12:11:59.921000+0000","last_peered":"2024-03-04T12:11:59.921000+0000","last_clean":"2024-03-04T12:11:59.921000+0000","last_became_active":"2024-03-04T12:11:58.939177+0000","last_became_peered":"2024-03-04T12:11:58.939177+0000","last_unstale":"2024-03-04T12:11:59.921000+0000","last_undegraded":"2024-03-04T12:11:59.921000+0000","last_fullsized":"2024-03-04T12:11:59.921000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:35.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:54.906271+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:54.905428+0000","last_became_peered":"2024-03-04T12:11:54.905428+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921284+0000","last_change":"2024-03-04T12:11:53.885617+0000","last_active":"2024-03-04T12:11:59.921284+0000","last_peered":"2024-03-04T12:11:59.921284+0000","last_clean":"2024-03-04T12:11:59.921284+0000","last_became_active":"2024-03-04T12:11:53.885329+0000","last_became_peered":"2024-03-04T12:11:53.885329+0000","last_unstale":"2024-03-04T12:11:59.921284+0000","last_undegraded":"2024-03-04T12:11:59.921284+0000","last_fullsized":"2024-03-04T12:11:59.921284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921212+0000","last_change":"2024-03-04T12:11:58.946535+0000","last_active":"2024-03-04T12:11:59.921212+0000","last_peered":"2024-03-04T12:11:59.921212+0000","last_clean":"2024-03-04T12:11:59.921212+0000","last_became_active":"2024-03-04T12:11:58.946055+0000","last_became_peered":"2024-03-04T12:11:58.946055+0000","last_unstale":"2024-03-04T12:11:59.921212+0000","last_undegraded":"2024-03-04T12:11:59.921212+0000","last_fullsized":"2024-03-04T12:11:59.921212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:46.739964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772109+0000","last_change":"2024-03-04T12:11:54.910474+0000","last_active":"2024-03-04T12:12:00.772109+0000","last_peered":"2024-03-04T12:12:00.772109+0000","last_clean":"2024-03-04T12:12:00.772109+0000","last_became_active":"2024-03-04T12:11:54.909806+0000","last_became_peered":"2024-03-04T12:11:54.909806+0000","last_unstale":"2024-03-04T12:12:00.772109+0000","last_undegraded":"2024-03-04T12:12:00.772109+0000","last_fullsized":"2024-03-04T12:12:00.772109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771668+0000","last_change":"2024-03-04T12:11:53.882414+0000","last_active":"2024-03-04T12:12:00.771668+0000","last_peered":"2024-03-04T12:12:00.771668+0000","last_clean":"2024-03-04T12:12:00.771668+0000","last_became_active":"2024-03-04T12:11:53.882147+0000","last_became_peered":"2024-03-04T12:11:53.882147+0000","last_unstale":"2024-03-04T12:12:00.771668+0000","last_undegraded":"2024-03-04T12:12:00.771668+0000","last_fullsized":"2024-03-04T12:12:00.771668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771019+0000","last_change":"2024-03-04T12:11:58.946384+0000","last_active":"2024-03-04T12:12:00.771019+0000","last_peered":"2024-03-04T12:12:00.771019+0000","last_clean":"2024-03-04T12:12:00.771019+0000","last_became_active":"2024-03-04T12:11:58.945989+0000","last_became_peered":"2024-03-04T12:11:58.945989+0000","last_unstale":"2024-03-04T12:12:00.771019+0000","last_undegraded":"2024-03-04T12:12:00.771019+0000","last_fullsized":"2024-03-04T12:12:00.771019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772496+0000","last_change":"2024-03-04T12:11:54.903131+0000","last_active":"2024-03-04T12:12:00.772496+0000","last_peered":"2024-03-04T12:12:00.772496+0000","last_clean":"2024-03-04T12:12:00.772496+0000","last_became_active":"2024-03-04T12:11:54.902729+0000","last_became_peered":"2024-03-04T12:11:54.902729+0000","last_unstale":"2024-03-04T12:12:00.772496+0000","last_undegraded":"2024-03-04T12:12:00.772496+0000","last_fullsized":"2024-03-04T12:12:00.772496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:43.148028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921116+0000","last_change":"2024-03-04T12:11:53.884623+0000","last_active":"2024-03-04T12:11:59.921116+0000","last_peered":"2024-03-04T12:11:59.921116+0000","last_clean":"2024-03-04T12:11:59.921116+0000","last_became_active":"2024-03-04T12:11:53.884222+0000","last_became_peered":"2024-03-04T12:11:53.884222+0000","last_unstale":"2024-03-04T12:11:59.921116+0000","last_undegraded":"2024-03-04T12:11:59.921116+0000","last_fullsized":"2024-03-04T12:11:59.921116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771150+0000","last_change":"2024-03-04T12:11:58.935385+0000","last_active":"2024-03-04T12:12:00.771150+0000","last_peered":"2024-03-04T12:12:00.771150+0000","last_clean":"2024-03-04T12:12:00.771150+0000","last_became_active":"2024-03-04T12:11:58.934991+0000","last_became_peered":"2024-03-04T12:11:58.934991+0000","last_unstale":"2024-03-04T12:12:00.771150+0000","last_undegraded":"2024-03-04T12:12:00.771150+0000","last_fullsized":"2024-03-04T12:12:00.771150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:09.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773010+0000","last_change":"2024-03-04T12:11:54.901559+0000","last_active":"2024-03-04T12:12:00.773010+0000","last_peered":"2024-03-04T12:12:00.773010+0000","last_clean":"2024-03-04T12:12:00.773010+0000","last_became_active":"2024-03-04T12:11:54.901215+0000","last_became_peered":"2024-03-04T12:11:54.901215+0000","last_unstale":"2024-03-04T12:12:00.773010+0000","last_undegraded":"2024-03-04T12:12:00.773010+0000","last_fullsized":"2024-03-04T12:12:00.773010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772226+0000","last_change":"2024-03-04T12:11:53.884589+0000","last_active":"2024-03-04T12:12:00.772226+0000","last_peered":"2024-03-04T12:12:00.772226+0000","last_clean":"2024-03-04T12:12:00.772226+0000","last_became_active":"2024-03-04T12:11:53.884279+0000","last_became_peered":"2024-03-04T12:11:53.884279+0000","last_unstale":"2024-03-04T12:12:00.772226+0000","last_undegraded":"2024-03-04T12:12:00.772226+0000","last_fullsized":"2024-03-04T12:12:00.772226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:54.931530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771590+0000","last_change":"2024-03-04T12:11:58.945665+0000","last_active":"2024-03-04T12:12:00.771590+0000","last_peered":"2024-03-04T12:12:00.771590+0000","last_clean":"2024-03-04T12:12:00.771590+0000","last_became_active":"2024-03-04T12:11:58.945297+0000","last_became_peered":"2024-03-04T12:11:58.945297+0000","last_unstale":"2024-03-04T12:12:00.771590+0000","last_undegraded":"2024-03-04T12:12:00.771590+0000","last_fullsized":"2024-03-04T12:12:00.771590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:11.565262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771183+0000","last_change":"2024-03-04T12:11:54.902251+0000","last_active":"2024-03-04T12:12:00.771183+0000","last_peered":"2024-03-04T12:12:00.771183+0000","last_clean":"2024-03-04T12:12:00.771183+0000","last_became_active":"2024-03-04T12:11:54.901942+0000","last_became_peered":"2024-03-04T12:11:54.901942+0000","last_unstale":"2024-03-04T12:12:00.771183+0000","last_undegraded":"2024-03-04T12:12:00.771183+0000","last_fullsized":"2024-03-04T12:12:00.771183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920811+0000","last_change":"2024-03-04T12:11:53.883599+0000","last_active":"2024-03-04T12:11:59.920811+0000","last_peered":"2024-03-04T12:11:59.920811+0000","last_clean":"2024-03-04T12:11:59.920811+0000","last_became_active":"2024-03-04T12:11:53.883317+0000","last_became_peered":"2024-03-04T12:11:53.883317+0000","last_unstale":"2024-03-04T12:11:59.920811+0000","last_undegraded":"2024-03-04T12:11:59.920811+0000","last_fullsized":"2024-03-04T12:11:59.920811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771562+0000","last_change":"2024-03-04T12:11:58.936436+0000","last_active":"2024-03-04T12:12:00.771562+0000","last_peered":"2024-03-04T12:12:00.771562+0000","last_clean":"2024-03-04T12:12:00.771562+0000","last_became_active":"2024-03-04T12:11:58.935593+0000","last_became_peered":"2024-03-04T12:11:58.935593+0000","last_unstale":"2024-03-04T12:12:00.771562+0000","last_undegraded":"2024-03-04T12:12:00.771562+0000","last_fullsized":"2024-03-04T12:12:00.771562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:29:47.188857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920930+0000","last_change":"2024-03-04T12:11:54.908033+0000","last_active":"2024-03-04T12:11:59.920930+0000","last_peered":"2024-03-04T12:11:59.920930+0000","last_clean":"2024-03-04T12:11:59.920930+0000","last_became_active":"2024-03-04T12:11:54.907784+0000","last_became_peered":"2024-03-04T12:11:54.907784+0000","last_unstale":"2024-03-04T12:11:59.920930+0000","last_undegraded":"2024-03-04T12:11:59.920930+0000","last_fullsized":"2024-03-04T12:11:59.920930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921086+0000","last_change":"2024-03-04T12:11:53.889166+0000","last_active":"2024-03-04T12:11:59.921086+0000","last_peered":"2024-03-04T12:11:59.921086+0000","last_clean":"2024-03-04T12:11:59.921086+0000","last_became_active":"2024-03-04T12:11:53.888906+0000","last_became_peered":"2024-03-04T12:11:53.888906+0000","last_unstale":"2024-03-04T12:11:59.921086+0000","last_undegraded":"2024-03-04T12:11:59.921086+0000","last_fullsized":"2024-03-04T12:11:59.921086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:56.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921204+0000","last_change":"2024-03-04T12:11:58.944078+0000","last_active":"2024-03-04T12:11:59.921204+0000","last_peered":"2024-03-04T12:11:59.921204+0000","last_clean":"2024-03-04T12:11:59.921204+0000","last_became_active":"2024-03-04T12:11:58.942753+0000","last_became_peered":"2024-03-04T12:11:58.942753+0000","last_unstale":"2024-03-04T12:11:59.921204+0000","last_undegraded":"2024-03-04T12:11:59.921204+0000","last_fullsized":"2024-03-04T12:11:59.921204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:35.014220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772005+0000","last_change":"2024-03-04T12:11:54.902835+0000","last_active":"2024-03-04T12:12:00.772005+0000","last_peered":"2024-03-04T12:12:00.772005+0000","last_clean":"2024-03-04T12:12:00.772005+0000","last_became_active":"2024-03-04T12:11:54.902550+0000","last_became_peered":"2024-03-04T12:11:54.902550+0000","last_unstale":"2024-03-04T12:12:00.772005+0000","last_undegraded":"2024-03-04T12:12:00.772005+0000","last_fullsized":"2024-03-04T12:12:00.772005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771751+0000","last_change":"2024-03-04T12:11:53.885828+0000","last_active":"2024-03-04T12:12:00.771751+0000","last_peered":"2024-03-04T12:12:00.771751+0000","last_clean":"2024-03-04T12:12:00.771751+0000","last_became_active":"2024-03-04T12:11:53.885526+0000","last_became_peered":"2024-03-04T12:11:53.885526+0000","last_unstale":"2024-03-04T12:12:00.771751+0000","last_undegraded":"2024-03-04T12:12:00.771751+0000","last_fullsized":"2024-03-04T12:12:00.771751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920921+0000","last_change":"2024-03-04T12:11:58.943710+0000","last_active":"2024-03-04T12:11:59.920921+0000","last_peered":"2024-03-04T12:11:59.920921+0000","last_clean":"2024-03-04T12:11:59.920921+0000","last_became_active":"2024-03-04T12:11:58.943297+0000","last_became_peered":"2024-03-04T12:11:58.943297+0000","last_unstale":"2024-03-04T12:11:59.920921+0000","last_undegraded":"2024-03-04T12:11:59.920921+0000","last_fullsized":"2024-03-04T12:11:59.920921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:56.606927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770672+0000","last_change":"2024-03-04T12:11:54.900415+0000","last_active":"2024-03-04T12:12:00.770672+0000","last_peered":"2024-03-04T12:12:00.770672+0000","last_clean":"2024-03-04T12:12:00.770672+0000","last_became_active":"2024-03-04T12:11:54.900101+0000","last_became_peered":"2024-03-04T12:11:54.900101+0000","last_unstale":"2024-03-04T12:12:00.770672+0000","last_undegraded":"2024-03-04T12:12:00.770672+0000","last_fullsized":"2024-03-04T12:12:00.770672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:33:07.725858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771029+0000","last_change":"2024-03-04T12:11:53.888493+0000","last_active":"2024-03-04T12:12:00.771029+0000","last_peered":"2024-03-04T12:12:00.771029+0000","last_clean":"2024-03-04T12:12:00.771029+0000","last_became_active":"2024-03-04T12:11:53.888223+0000","last_became_peered":"2024-03-04T12:11:53.888223+0000","last_unstale":"2024-03-04T12:12:00.771029+0000","last_undegraded":"2024-03-04T12:12:00.771029+0000","last_fullsized":"2024-03-04T12:12:00.771029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:40.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772694+0000","last_change":"2024-03-04T12:11:58.934500+0000","last_active":"2024-03-04T12:12:00.772694+0000","last_peered":"2024-03-04T12:12:00.772694+0000","last_clean":"2024-03-04T12:12:00.772694+0000","last_became_active":"2024-03-04T12:11:58.933910+0000","last_became_peered":"2024-03-04T12:11:58.933910+0000","last_unstale":"2024-03-04T12:12:00.772694+0000","last_undegraded":"2024-03-04T12:12:00.772694+0000","last_fullsized":"2024-03-04T12:12:00.772694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:42:29.560174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771718+0000","last_change":"2024-03-04T12:11:54.906835+0000","last_active":"2024-03-04T12:12:00.771718+0000","last_peered":"2024-03-04T12:12:00.771718+0000","last_clean":"2024-03-04T12:12:00.771718+0000","last_became_active":"2024-03-04T12:11:54.904477+0000","last_became_peered":"2024-03-04T12:11:54.904477+0000","last_unstale":"2024-03-04T12:12:00.771718+0000","last_undegraded":"2024-03-04T12:12:00.771718+0000","last_fullsized":"2024-03-04T12:12:00.771718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:38.104123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:04.123314+0000","last_change":"2024-03-04T12:11:53.889752+0000","last_active":"2024-03-04T12:12:04.123314+0000","last_peered":"2024-03-04T12:12:04.123314+0000","last_clean":"2024-03-04T12:12:04.123314+0000","last_became_active":"2024-03-04T12:11:53.889469+0000","last_became_peered":"2024-03-04T12:11:53.889469+0000","last_unstale":"2024-03-04T12:12:04.123314+0000","last_undegraded":"2024-03-04T12:12:04.123314+0000","last_fullsized":"2024-03-04T12:12:04.123314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771293+0000","last_change":"2024-03-04T12:11:58.941223+0000","last_active":"2024-03-04T12:12:00.771293+0000","last_peered":"2024-03-04T12:12:00.771293+0000","last_clean":"2024-03-04T12:12:00.771293+0000","last_became_active":"2024-03-04T12:11:58.940845+0000","last_became_peered":"2024-03-04T12:11:58.940845+0000","last_unstale":"2024-03-04T12:12:00.771293+0000","last_undegraded":"2024-03-04T12:12:00.771293+0000","last_fullsized":"2024-03-04T12:12:00.771293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:48.020894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772686+0000","last_change":"2024-03-04T12:11:54.902174+0000","last_active":"2024-03-04T12:12:00.772686+0000","last_peered":"2024-03-04T12:12:00.772686+0000","last_clean":"2024-03-04T12:12:00.772686+0000","last_became_active":"2024-03-04T12:11:54.901884+0000","last_became_peered":"2024-03-04T12:11:54.901884+0000","last_unstale":"2024-03-04T12:12:00.772686+0000","last_undegraded":"2024-03-04T12:12:00.772686+0000","last_fullsized":"2024-03-04T12:12:00.772686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771560+0000","last_change":"2024-03-04T12:11:53.892494+0000","last_active":"2024-03-04T12:12:00.771560+0000","last_peered":"2024-03-04T12:12:00.771560+0000","last_clean":"2024-03-04T12:12:00.771560+0000","last_became_active":"2024-03-04T12:11:53.892170+0000","last_became_peered":"2024-03-04T12:11:53.892170+0000","last_unstale":"2024-03-04T12:12:00.771560+0000","last_undegraded":"2024-03-04T12:12:00.771560+0000","last_fullsized":"2024-03-04T12:12:00.771560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773086+0000","last_change":"2024-03-04T12:11:58.945966+0000","last_active":"2024-03-04T12:12:00.773086+0000","last_peered":"2024-03-04T12:12:00.773086+0000","last_clean":"2024-03-04T12:12:00.773086+0000","last_became_active":"2024-03-04T12:11:58.945545+0000","last_became_peered":"2024-03-04T12:11:58.945545+0000","last_unstale":"2024-03-04T12:12:00.773086+0000","last_undegraded":"2024-03-04T12:12:00.773086+0000","last_fullsized":"2024-03-04T12:12:00.773086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771604+0000","last_change":"2024-03-04T12:11:54.900788+0000","last_active":"2024-03-04T12:12:00.771604+0000","last_peered":"2024-03-04T12:12:00.771604+0000","last_clean":"2024-03-04T12:12:00.771604+0000","last_became_active":"2024-03-04T12:11:54.900523+0000","last_became_peered":"2024-03-04T12:11:54.900523+0000","last_unstale":"2024-03-04T12:12:00.771604+0000","last_undegraded":"2024-03-04T12:12:00.771604+0000","last_fullsized":"2024-03-04T12:12:00.771604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:51:07.524433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773050+0000","last_change":"2024-03-04T12:11:53.887458+0000","last_active":"2024-03-04T12:12:00.773050+0000","last_peered":"2024-03-04T12:12:00.773050+0000","last_clean":"2024-03-04T12:12:00.773050+0000","last_became_active":"2024-03-04T12:11:53.887205+0000","last_became_peered":"2024-03-04T12:11:53.887205+0000","last_unstale":"2024-03-04T12:12:00.773050+0000","last_undegraded":"2024-03-04T12:12:00.773050+0000","last_fullsized":"2024-03-04T12:12:00.773050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920213+0000","last_change":"2024-03-04T12:11:58.944437+0000","last_active":"2024-03-04T12:11:59.920213+0000","last_peered":"2024-03-04T12:11:59.920213+0000","last_clean":"2024-03-04T12:11:59.920213+0000","last_became_active":"2024-03-04T12:11:58.944027+0000","last_became_peered":"2024-03-04T12:11:58.944027+0000","last_unstale":"2024-03-04T12:11:59.920213+0000","last_undegraded":"2024-03-04T12:11:59.920213+0000","last_fullsized":"2024-03-04T12:11:59.920213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:09:47.849893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.773158+0000","last_change":"2024-03-04T12:11:54.904517+0000","last_active":"2024-03-04T12:12:00.773158+0000","last_peered":"2024-03-04T12:12:00.773158+0000","last_clean":"2024-03-04T12:12:00.773158+0000","last_became_active":"2024-03-04T12:11:54.904245+0000","last_became_peered":"2024-03-04T12:11:54.904245+0000","last_unstale":"2024-03-04T12:12:00.773158+0000","last_undegraded":"2024-03-04T12:12:00.773158+0000","last_fullsized":"2024-03-04T12:12:00.773158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921498+0000","last_change":"2024-03-04T12:11:53.888433+0000","last_active":"2024-03-04T12:11:59.921498+0000","last_peered":"2024-03-04T12:11:59.921498+0000","last_clean":"2024-03-04T12:11:59.921498+0000","last_became_active":"2024-03-04T12:11:53.888164+0000","last_became_peered":"2024-03-04T12:11:53.888164+0000","last_unstale":"2024-03-04T12:11:59.921498+0000","last_undegraded":"2024-03-04T12:11:59.921498+0000","last_fullsized":"2024-03-04T12:11:59.921498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920282+0000","last_change":"2024-03-04T12:11:58.937226+0000","last_active":"2024-03-04T12:11:59.920282+0000","last_peered":"2024-03-04T12:11:59.920282+0000","last_clean":"2024-03-04T12:11:59.920282+0000","last_became_active":"2024-03-04T12:11:58.936831+0000","last_became_peered":"2024-03-04T12:11:58.936831+0000","last_unstale":"2024-03-04T12:11:59.920282+0000","last_undegraded":"2024-03-04T12:11:59.920282+0000","last_fullsized":"2024-03-04T12:11:59.920282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:02.110563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770587+0000","last_change":"2024-03-04T12:11:58.946973+0000","last_active":"2024-03-04T12:12:00.770587+0000","last_peered":"2024-03-04T12:12:00.770587+0000","last_clean":"2024-03-04T12:12:00.770587+0000","last_became_active":"2024-03-04T12:11:58.946581+0000","last_became_peered":"2024-03-04T12:11:58.946581+0000","last_unstale":"2024-03-04T12:12:00.770587+0000","last_undegraded":"2024-03-04T12:12:00.770587+0000","last_fullsized":"2024-03-04T12:12:00.770587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:48.809461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770784+0000","last_change":"2024-03-04T12:11:54.900431+0000","last_active":"2024-03-04T12:12:00.770784+0000","last_peered":"2024-03-04T12:12:00.770784+0000","last_clean":"2024-03-04T12:12:00.770784+0000","last_became_active":"2024-03-04T12:11:54.900166+0000","last_became_peered":"2024-03-04T12:11:54.900166+0000","last_unstale":"2024-03-04T12:12:00.770784+0000","last_undegraded":"2024-03-04T12:12:00.770784+0000","last_fullsized":"2024-03-04T12:12:00.770784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:53.879757+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:53.878758+0000","last_became_peered":"2024-03-04T12:11:53.878758+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:54:22.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921468+0000","last_change":"2024-03-04T12:11:58.945414+0000","last_active":"2024-03-04T12:11:59.921468+0000","last_peered":"2024-03-04T12:11:59.921468+0000","last_clean":"2024-03-04T12:11:59.921468+0000","last_became_active":"2024-03-04T12:11:58.945031+0000","last_became_peered":"2024-03-04T12:11:58.945031+0000","last_unstale":"2024-03-04T12:11:59.921468+0000","last_undegraded":"2024-03-04T12:11:59.921468+0000","last_fullsized":"2024-03-04T12:11:59.921468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:42.144953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920217+0000","last_change":"2024-03-04T12:11:54.896991+0000","last_active":"2024-03-04T12:11:59.920217+0000","last_peered":"2024-03-04T12:11:59.920217+0000","last_clean":"2024-03-04T12:11:59.920217+0000","last_became_active":"2024-03-04T12:11:54.896725+0000","last_became_peered":"2024-03-04T12:11:54.896725+0000","last_unstale":"2024-03-04T12:11:59.920217+0000","last_undegraded":"2024-03-04T12:11:59.920217+0000","last_fullsized":"2024-03-04T12:11:59.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920530+0000","last_change":"2024-03-04T12:11:53.884328+0000","last_active":"2024-03-04T12:11:59.920530+0000","last_peered":"2024-03-04T12:11:59.920530+0000","last_clean":"2024-03-04T12:11:59.920530+0000","last_became_active":"2024-03-04T12:11:53.882865+0000","last_became_peered":"2024-03-04T12:11:53.882865+0000","last_unstale":"2024-03-04T12:11:59.920530+0000","last_undegraded":"2024-03-04T12:11:59.920530+0000","last_fullsized":"2024-03-04T12:11:59.920530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:34:30.159611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921283+0000","last_change":"2024-03-04T12:11:53.881932+0000","last_active":"2024-03-04T12:11:59.921283+0000","last_peered":"2024-03-04T12:11:59.921283+0000","last_clean":"2024-03-04T12:11:59.921283+0000","last_became_active":"2024-03-04T12:11:53.880843+0000","last_became_peered":"2024-03-04T12:11:53.880843+0000","last_unstale":"2024-03-04T12:11:59.921283+0000","last_undegraded":"2024-03-04T12:11:59.921283+0000","last_fullsized":"2024-03-04T12:11:59.921283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:35.233350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770941+0000","last_change":"2024-03-04T12:11:54.892168+0000","last_active":"2024-03-04T12:12:00.770941+0000","last_peered":"2024-03-04T12:12:00.770941+0000","last_clean":"2024-03-04T12:12:00.770941+0000","last_became_active":"2024-03-04T12:11:54.891886+0000","last_became_peered":"2024-03-04T12:11:54.891886+0000","last_unstale":"2024-03-04T12:12:00.770941+0000","last_undegraded":"2024-03-04T12:12:00.770941+0000","last_fullsized":"2024-03-04T12:12:00.770941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920785+0000","last_change":"2024-03-04T12:11:58.946859+0000","last_active":"2024-03-04T12:11:59.920785+0000","last_peered":"2024-03-04T12:11:59.920785+0000","last_clean":"2024-03-04T12:11:59.920785+0000","last_became_active":"2024-03-04T12:11:58.946495+0000","last_became_peered":"2024-03-04T12:11:58.946495+0000","last_unstale":"2024-03-04T12:11:59.920785+0000","last_undegraded":"2024-03-04T12:11:59.920785+0000","last_fullsized":"2024-03-04T12:11:59.920785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:25.320885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920580+0000","last_change":"2024-03-04T12:11:53.889151+0000","last_active":"2024-03-04T12:11:59.920580+0000","last_peered":"2024-03-04T12:11:59.920580+0000","last_clean":"2024-03-04T12:11:59.920580+0000","last_became_active":"2024-03-04T12:11:53.888882+0000","last_became_peered":"2024-03-04T12:11:53.888882+0000","last_unstale":"2024-03-04T12:11:59.920580+0000","last_undegraded":"2024-03-04T12:11:59.920580+0000","last_fullsized":"2024-03-04T12:11:59.920580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:52.862034+0000","last_clean_scrub_stamp":"2024-03-04T12:11:52.862034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:55.601777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772788+0000","last_change":"2024-03-04T12:11:54.901800+0000","last_active":"2024-03-04T12:12:00.772788+0000","last_peered":"2024-03-04T12:12:00.772788+0000","last_clean":"2024-03-04T12:12:00.772788+0000","last_became_active":"2024-03-04T12:11:54.901502+0000","last_became_peered":"2024-03-04T12:11:54.901502+0000","last_unstale":"2024-03-04T12:12:00.772788+0000","last_undegraded":"2024-03-04T12:12:00.772788+0000","last_fullsized":"2024-03-04T12:12:00.772788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:36.238592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921109+0000","last_change":"2024-03-04T12:11:58.943607+0000","last_active":"2024-03-04T12:11:59.921109+0000","last_peered":"2024-03-04T12:11:59.921109+0000","last_clean":"2024-03-04T12:11:59.921109+0000","last_became_active":"2024-03-04T12:11:58.942837+0000","last_became_peered":"2024-03-04T12:11:58.942837+0000","last_unstale":"2024-03-04T12:11:59.921109+0000","last_undegraded":"2024-03-04T12:11:59.921109+0000","last_fullsized":"2024-03-04T12:11:59.921109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:57.908105+0000","last_clean_scrub_stamp":"2024-03-04T12:11:57.908105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:59.670150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920801+0000","last_change":"2024-03-04T12:11:54.908034+0000","last_active":"2024-03-04T12:11:59.920801+0000","last_peered":"2024-03-04T12:11:59.920801+0000","last_clean":"2024-03-04T12:11:59.920801+0000","last_became_active":"2024-03-04T12:11:54.907761+0000","last_became_peered":"2024-03-04T12:11:54.907761+0000","last_unstale":"2024-03-04T12:11:59.920801+0000","last_undegraded":"2024-03-04T12:11:59.920801+0000","last_fullsized":"2024-03-04T12:11:59.920801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921098+0000","last_change":"2024-03-04T12:11:54.904118+0000","last_active":"2024-03-04T12:11:59.921098+0000","last_peered":"2024-03-04T12:11:59.921098+0000","last_clean":"2024-03-04T12:11:59.921098+0000","last_became_active":"2024-03-04T12:11:54.903747+0000","last_became_peered":"2024-03-04T12:11:54.903747+0000","last_unstale":"2024-03-04T12:11:59.921098+0000","last_undegraded":"2024-03-04T12:11:59.921098+0000","last_fullsized":"2024-03-04T12:11:59.921098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:24:55.335318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771638+0000","last_change":"2024-03-04T12:11:54.900197+0000","last_active":"2024-03-04T12:12:00.771638+0000","last_peered":"2024-03-04T12:12:00.771638+0000","last_clean":"2024-03-04T12:12:00.771638+0000","last_became_active":"2024-03-04T12:11:54.899939+0000","last_became_peered":"2024-03-04T12:11:54.899939+0000","last_unstale":"2024-03-04T12:12:00.771638+0000","last_undegraded":"2024-03-04T12:12:00.771638+0000","last_fullsized":"2024-03-04T12:12:00.771638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770848+0000","last_change":"2024-03-04T12:11:54.896614+0000","last_active":"2024-03-04T12:12:00.770848+0000","last_peered":"2024-03-04T12:12:00.770848+0000","last_clean":"2024-03-04T12:12:00.770848+0000","last_became_active":"2024-03-04T12:11:54.896312+0000","last_became_peered":"2024-03-04T12:11:54.896312+0000","last_unstale":"2024-03-04T12:12:00.770848+0000","last_undegraded":"2024-03-04T12:12:00.770848+0000","last_fullsized":"2024-03-04T12:12:00.770848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920990+0000","last_change":"2024-03-04T12:11:54.898597+0000","last_active":"2024-03-04T12:11:59.920990+0000","last_peered":"2024-03-04T12:11:59.920990+0000","last_clean":"2024-03-04T12:11:59.920990+0000","last_became_active":"2024-03-04T12:11:54.898339+0000","last_became_peered":"2024-03-04T12:11:54.898339+0000","last_unstale":"2024-03-04T12:11:59.920990+0000","last_undegraded":"2024-03-04T12:11:59.920990+0000","last_fullsized":"2024-03-04T12:11:59.920990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771385+0000","last_change":"2024-03-04T12:11:54.902225+0000","last_active":"2024-03-04T12:12:00.771385+0000","last_peered":"2024-03-04T12:12:00.771385+0000","last_clean":"2024-03-04T12:12:00.771385+0000","last_became_active":"2024-03-04T12:11:54.901968+0000","last_became_peered":"2024-03-04T12:11:54.901968+0000","last_unstale":"2024-03-04T12:12:00.771385+0000","last_undegraded":"2024-03-04T12:12:00.771385+0000","last_fullsized":"2024-03-04T12:12:00.771385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921621+0000","last_change":"2024-03-04T12:11:54.911542+0000","last_active":"2024-03-04T12:11:59.921621+0000","last_peered":"2024-03-04T12:11:59.921621+0000","last_clean":"2024-03-04T12:11:59.921621+0000","last_became_active":"2024-03-04T12:11:54.911032+0000","last_became_peered":"2024-03-04T12:11:54.911032+0000","last_unstale":"2024-03-04T12:11:59.921621+0000","last_undegraded":"2024-03-04T12:11:59.921621+0000","last_fullsized":"2024-03-04T12:11:59.921621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921271+0000","last_change":"2024-03-04T12:11:54.897959+0000","last_active":"2024-03-04T12:11:59.921271+0000","last_peered":"2024-03-04T12:11:59.921271+0000","last_clean":"2024-03-04T12:11:59.921271+0000","last_became_active":"2024-03-04T12:11:54.897665+0000","last_became_peered":"2024-03-04T12:11:54.897665+0000","last_unstale":"2024-03-04T12:11:59.921271+0000","last_undegraded":"2024-03-04T12:11:59.921271+0000","last_fullsized":"2024-03-04T12:11:59.921271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771216+0000","last_change":"2024-03-04T12:11:54.899561+0000","last_active":"2024-03-04T12:12:00.771216+0000","last_peered":"2024-03-04T12:12:00.771216+0000","last_clean":"2024-03-04T12:12:00.771216+0000","last_became_active":"2024-03-04T12:11:54.899289+0000","last_became_peered":"2024-03-04T12:11:54.899289+0000","last_unstale":"2024-03-04T12:12:00.771216+0000","last_undegraded":"2024-03-04T12:12:00.771216+0000","last_fullsized":"2024-03-04T12:12:00.771216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920409+0000","last_change":"2024-03-04T12:11:54.897860+0000","last_active":"2024-03-04T12:11:59.920409+0000","last_peered":"2024-03-04T12:11:59.920409+0000","last_clean":"2024-03-04T12:11:59.920409+0000","last_became_active":"2024-03-04T12:11:54.897585+0000","last_became_peered":"2024-03-04T12:11:54.897585+0000","last_unstale":"2024-03-04T12:11:59.920409+0000","last_undegraded":"2024-03-04T12:11:59.920409+0000","last_fullsized":"2024-03-04T12:11:59.920409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771542+0000","last_change":"2024-03-04T12:11:54.904095+0000","last_active":"2024-03-04T12:12:00.771542+0000","last_peered":"2024-03-04T12:12:00.771542+0000","last_clean":"2024-03-04T12:12:00.771542+0000","last_became_active":"2024-03-04T12:11:54.903527+0000","last_became_peered":"2024-03-04T12:11:54.903527+0000","last_unstale":"2024-03-04T12:12:00.771542+0000","last_undegraded":"2024-03-04T12:12:00.771542+0000","last_fullsized":"2024-03-04T12:12:00.771542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:41:53.111115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770740+0000","last_change":"2024-03-04T12:11:54.904758+0000","last_active":"2024-03-04T12:12:00.770740+0000","last_peered":"2024-03-04T12:12:00.770740+0000","last_clean":"2024-03-04T12:12:00.770740+0000","last_became_active":"2024-03-04T12:11:54.904480+0000","last_became_peered":"2024-03-04T12:11:54.904480+0000","last_unstale":"2024-03-04T12:12:00.770740+0000","last_undegraded":"2024-03-04T12:12:00.770740+0000","last_fullsized":"2024-03-04T12:12:00.770740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921331+0000","last_change":"2024-03-04T12:11:54.901144+0000","last_active":"2024-03-04T12:11:59.921331+0000","last_peered":"2024-03-04T12:11:59.921331+0000","last_clean":"2024-03-04T12:11:59.921331+0000","last_became_active":"2024-03-04T12:11:54.900739+0000","last_became_peered":"2024-03-04T12:11:54.900739+0000","last_unstale":"2024-03-04T12:11:59.921331+0000","last_undegraded":"2024-03-04T12:11:59.921331+0000","last_fullsized":"2024-03-04T12:11:59.921331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:37.637332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771914+0000","last_change":"2024-03-04T12:11:54.905995+0000","last_active":"2024-03-04T12:12:00.771914+0000","last_peered":"2024-03-04T12:12:00.771914+0000","last_clean":"2024-03-04T12:12:00.771914+0000","last_became_active":"2024-03-04T12:11:54.904878+0000","last_became_peered":"2024-03-04T12:11:54.904878+0000","last_unstale":"2024-03-04T12:12:00.771914+0000","last_undegraded":"2024-03-04T12:12:00.771914+0000","last_fullsized":"2024-03-04T12:12:00.771914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:28:39.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921040+0000","last_change":"2024-03-04T12:11:54.904652+0000","last_active":"2024-03-04T12:11:59.921040+0000","last_peered":"2024-03-04T12:11:59.921040+0000","last_clean":"2024-03-04T12:11:59.921040+0000","last_became_active":"2024-03-04T12:11:54.904348+0000","last_became_peered":"2024-03-04T12:11:54.904348+0000","last_unstale":"2024-03-04T12:11:59.921040+0000","last_undegraded":"2024-03-04T12:11:59.921040+0000","last_fullsized":"2024-03-04T12:11:59.921040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:58:05.687532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772261+0000","last_change":"2024-03-04T12:11:54.905800+0000","last_active":"2024-03-04T12:12:00.772261+0000","last_peered":"2024-03-04T12:12:00.772261+0000","last_clean":"2024-03-04T12:12:00.772261+0000","last_became_active":"2024-03-04T12:11:54.905550+0000","last_became_peered":"2024-03-04T12:11:54.905550+0000","last_unstale":"2024-03-04T12:12:00.772261+0000","last_undegraded":"2024-03-04T12:12:00.772261+0000","last_fullsized":"2024-03-04T12:12:00.772261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:44.768632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771115+0000","last_change":"2024-03-04T12:11:54.909971+0000","last_active":"2024-03-04T12:12:00.771115+0000","last_peered":"2024-03-04T12:12:00.771115+0000","last_clean":"2024-03-04T12:12:00.771115+0000","last_became_active":"2024-03-04T12:11:54.909694+0000","last_became_peered":"2024-03-04T12:11:54.909694+0000","last_unstale":"2024-03-04T12:12:00.771115+0000","last_undegraded":"2024-03-04T12:12:00.771115+0000","last_fullsized":"2024-03-04T12:12:00.771115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770786+0000","last_change":"2024-03-04T12:11:54.901482+0000","last_active":"2024-03-04T12:12:00.770786+0000","last_peered":"2024-03-04T12:12:00.770786+0000","last_clean":"2024-03-04T12:12:00.770786+0000","last_became_active":"2024-03-04T12:11:54.901185+0000","last_became_peered":"2024-03-04T12:11:54.901185+0000","last_unstale":"2024-03-04T12:12:00.770786+0000","last_undegraded":"2024-03-04T12:12:00.770786+0000","last_fullsized":"2024-03-04T12:12:00.770786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.770638+0000","last_change":"2024-03-04T12:11:54.902348+0000","last_active":"2024-03-04T12:12:00.770638+0000","last_peered":"2024-03-04T12:12:00.770638+0000","last_clean":"2024-03-04T12:12:00.770638+0000","last_became_active":"2024-03-04T12:11:54.902087+0000","last_became_peered":"2024-03-04T12:11:54.902087+0000","last_unstale":"2024-03-04T12:12:00.770638+0000","last_undegraded":"2024-03-04T12:12:00.770638+0000","last_fullsized":"2024-03-04T12:12:00.770638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:04:00.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920436+0000","last_change":"2024-03-04T12:11:54.904336+0000","last_active":"2024-03-04T12:11:59.920436+0000","last_peered":"2024-03-04T12:11:59.920436+0000","last_clean":"2024-03-04T12:11:59.920436+0000","last_became_active":"2024-03-04T12:11:54.904049+0000","last_became_peered":"2024-03-04T12:11:54.904049+0000","last_unstale":"2024-03-04T12:11:59.920436+0000","last_undegraded":"2024-03-04T12:11:59.920436+0000","last_fullsized":"2024-03-04T12:11:59.920436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:42.699135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772596+0000","last_change":"2024-03-04T12:11:54.907818+0000","last_active":"2024-03-04T12:12:00.772596+0000","last_peered":"2024-03-04T12:12:00.772596+0000","last_clean":"2024-03-04T12:12:00.772596+0000","last_became_active":"2024-03-04T12:11:54.907443+0000","last_became_peered":"2024-03-04T12:11:54.907443+0000","last_unstale":"2024-03-04T12:12:00.772596+0000","last_undegraded":"2024-03-04T12:12:00.772596+0000","last_fullsized":"2024-03-04T12:12:00.772596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:26.417645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921105+0000","last_change":"2024-03-04T12:11:54.906714+0000","last_active":"2024-03-04T12:11:59.921105+0000","last_peered":"2024-03-04T12:11:59.921105+0000","last_clean":"2024-03-04T12:11:59.921105+0000","last_became_active":"2024-03-04T12:11:54.906367+0000","last_became_peered":"2024-03-04T12:11:54.906367+0000","last_unstale":"2024-03-04T12:11:59.921105+0000","last_undegraded":"2024-03-04T12:11:59.921105+0000","last_fullsized":"2024-03-04T12:11:59.921105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:41:13.707483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.921624+0000","last_change":"2024-03-04T12:11:54.908667+0000","last_active":"2024-03-04T12:11:59.921624+0000","last_peered":"2024-03-04T12:11:59.921624+0000","last_clean":"2024-03-04T12:11:59.921624+0000","last_became_active":"2024-03-04T12:11:54.908377+0000","last_became_peered":"2024-03-04T12:11:54.908377+0000","last_unstale":"2024-03-04T12:11:59.921624+0000","last_undegraded":"2024-03-04T12:11:59.921624+0000","last_fullsized":"2024-03-04T12:11:59.921624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772398+0000","last_change":"2024-03-04T12:11:54.907669+0000","last_active":"2024-03-04T12:12:00.772398+0000","last_peered":"2024-03-04T12:12:00.772398+0000","last_clean":"2024-03-04T12:12:00.772398+0000","last_became_active":"2024-03-04T12:11:54.905673+0000","last_became_peered":"2024-03-04T12:11:54.905673+0000","last_unstale":"2024-03-04T12:12:00.772398+0000","last_undegraded":"2024-03-04T12:12:00.772398+0000","last_fullsized":"2024-03-04T12:12:00.772398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:43.980065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:11:59.920103+0000","last_change":"2024-03-04T12:11:54.904733+0000","last_active":"2024-03-04T12:11:59.920103+0000","last_peered":"2024-03-04T12:11:59.920103+0000","last_clean":"2024-03-04T12:11:59.920103+0000","last_became_active":"2024-03-04T12:11:54.903680+0000","last_became_peered":"2024-03-04T12:11:54.903680+0000","last_unstale":"2024-03-04T12:11:59.920103+0000","last_undegraded":"2024-03-04T12:11:59.920103+0000","last_fullsized":"2024-03-04T12:11:59.920103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:54.770107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771078+0000","last_change":"2024-03-04T12:11:54.902367+0000","last_active":"2024-03-04T12:12:00.771078+0000","last_peered":"2024-03-04T12:12:00.771078+0000","last_clean":"2024-03-04T12:12:00.771078+0000","last_became_active":"2024-03-04T12:11:54.902075+0000","last_became_peered":"2024-03-04T12:11:54.902075+0000","last_unstale":"2024-03-04T12:12:00.771078+0000","last_undegraded":"2024-03-04T12:12:00.771078+0000","last_fullsized":"2024-03-04T12:12:00.771078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:44.954556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771398+0000","last_change":"2024-03-04T12:11:54.899399+0000","last_active":"2024-03-04T12:12:00.771398+0000","last_peered":"2024-03-04T12:12:00.771398+0000","last_clean":"2024-03-04T12:12:00.771398+0000","last_became_active":"2024-03-04T12:11:54.899084+0000","last_became_peered":"2024-03-04T12:11:54.899084+0000","last_unstale":"2024-03-04T12:12:00.771398+0000","last_undegraded":"2024-03-04T12:12:00.771398+0000","last_fullsized":"2024-03-04T12:12:00.771398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:02:56.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772929+0000","last_change":"2024-03-04T12:11:54.903214+0000","last_active":"2024-03-04T12:12:00.772929+0000","last_peered":"2024-03-04T12:12:00.772929+0000","last_clean":"2024-03-04T12:12:00.772929+0000","last_became_active":"2024-03-04T12:11:54.901705+0000","last_became_peered":"2024-03-04T12:11:54.901705+0000","last_unstale":"2024-03-04T12:12:00.772929+0000","last_undegraded":"2024-03-04T12:12:00.772929+0000","last_fullsized":"2024-03-04T12:12:00.772929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:44:31.320982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771893+0000","last_change":"2024-03-04T12:11:54.910691+0000","last_active":"2024-03-04T12:12:00.771893+0000","last_peered":"2024-03-04T12:12:00.771893+0000","last_clean":"2024-03-04T12:12:00.771893+0000","last_became_active":"2024-03-04T12:11:54.910198+0000","last_became_peered":"2024-03-04T12:11:54.910198+0000","last_unstale":"2024-03-04T12:12:00.771893+0000","last_undegraded":"2024-03-04T12:12:00.771893+0000","last_fullsized":"2024-03-04T12:12:00.771893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771263+0000","last_change":"2024-03-04T12:11:54.898528+0000","last_active":"2024-03-04T12:12:00.771263+0000","last_peered":"2024-03-04T12:12:00.771263+0000","last_clean":"2024-03-04T12:12:00.771263+0000","last_became_active":"2024-03-04T12:11:54.897521+0000","last_became_peered":"2024-03-04T12:11:54.897521+0000","last_unstale":"2024-03-04T12:12:00.771263+0000","last_undegraded":"2024-03-04T12:12:00.771263+0000","last_fullsized":"2024-03-04T12:12:00.771263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:21:09.788925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.771266+0000","last_change":"2024-03-04T12:11:54.901718+0000","last_active":"2024-03-04T12:12:00.771266+0000","last_peered":"2024-03-04T12:12:00.771266+0000","last_clean":"2024-03-04T12:12:00.771266+0000","last_became_active":"2024-03-04T12:11:54.901395+0000","last_became_peered":"2024-03-04T12:11:54.901395+0000","last_unstale":"2024-03-04T12:12:00.771266+0000","last_undegraded":"2024-03-04T12:12:00.771266+0000","last_fullsized":"2024-03-04T12:12:00.771266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:46.611323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-04T12:12:00.772905+0000","last_change":"2024-03-04T12:11:54.907257+0000","last_active":"2024-03-04T12:12:00.772905+0000","last_peered":"2024-03-04T12:12:00.772905+0000","last_clean":"2024-03-04T12:12:00.772905+0000","last_became_active":"2024-03-04T12:11:54.906648+0000","last_became_peered":"2024-03-04T12:11:54.906648+0000","last_unstale":"2024-03-04T12:12:00.772905+0000","last_undegraded":"2024-03-04T12:12:00.772905+0000","last_fullsized":"2024-03-04T12:12:00.772905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:53.867276+0000","last_clean_scrub_stamp":"2024-03-04T12:11:53.867276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:02.739456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:12:22.585 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-04T12:12:22.586 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T12:12:22.586 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-04T12:12:22.586 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-04T12:12:22.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.719+0000 7f3aa40a1640 1 Processor -- start 2024-03-04T12:12:22.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.719+0000 7f3aa40a1640 1 -- start start 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa40a1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a9c158410 0x7f3a9c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa40a1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c06b500 con 0x7f3a9c06acd0 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa40a1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c06b7e0 con 0x7f3a9c161e70 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa40a1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b3f00 con 0x7f3a9c158410 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa1e3d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a9c158410 0x7f3a9c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa163c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3a9c06acd0 0x7f3a9c161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42516/0 (socket says 172.21.15.148:42516) 2024-03-04T12:12:22.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.720+0000 7f3aa163c640 1 -- 172.21.15.148:0/300032794 learned_addr learned my addr 172.21.15.148:0/300032794 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:22.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa1e3d640 1 -- 172.21.15.148:0/300032794 >> v1:172.21.15.191:6789/0 conn(0x7f3a9c161e70 legacy=0x7f3a9c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:22.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa1e3d640 1 -- 172.21.15.148:0/300032794 >> v1:172.21.15.148:6789/0 conn(0x7f3a9c06acd0 legacy=0x7f3a9c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa1e3d640 1 -- 172.21.15.148:0/300032794 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c06bac0 con 0x7f3a9c158410 2024-03-04T12:12:22.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa0e3b640 1 -- 172.21.15.148:0/300032794 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346296758 0 0) 0x7f3a9c06b500 con 0x7f3a9c06acd0 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa1e3d640 1 --2- 172.21.15.148:0/300032794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0b39c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b362aea95b8ed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.721+0000 7f3aa0e3b640 1 -- 172.21.15.148:0/300032794 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98034070 con 0x7f3a9c158410 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa0e3b640 1 -- 172.21.15.148:0/300032794 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98030610 con 0x7f3a9c158410 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa0e3b640 1 -- 172.21.15.148:0/300032794 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980309e0 con 0x7f3a9c158410 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 -- 172.21.15.148:0/300032794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 msgr2=0x7f3a9c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/300032794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0b39c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802b960 comp rx=0 tx=0).stop 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 -- 172.21.15.148:0/300032794 shutdown_connections 2024-03-04T12:12:22.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/300032794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0b39c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 -- 172.21.15.148:0/300032794 >> 172.21.15.148:0/300032794 conn(0x7f3a9c0cfef0 msgr2=0x7f3a9c0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 -- 172.21.15.148:0/300032794 shutdown_connections 2024-03-04T12:12:22.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.722+0000 7f3aa40a1640 1 -- 172.21.15.148:0/300032794 wait complete. 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.723+0000 7f3aa40a1640 1 Processor -- start 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.723+0000 7f3aa40a1640 1 -- start start 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a9c06acd0 0x7f3a9c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 0x7f3a9c0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c06c5a0 con 0x7f3a9c158410 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c06c080 con 0x7f3a9c06acd0 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa40a1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c06b160 con 0x7f3a9c161e70 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45260/0 (socket says 172.21.15.148:45260) 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 -- 172.21.15.148:0/480694278 learned_addr learned my addr 172.21.15.148:0/480694278 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa1e3d640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a9c06acd0 0x7f3a9c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa263e640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 0x7f3a9c0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 -- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 msgr2=0x7f3a9c0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 0x7f3a9c0ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 -- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a9c06acd0 msgr2=0x7f3a9c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a9c06acd0 0x7f3a9c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa163c640 1 -- 172.21.15.148:0/480694278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0cdbc0 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.724+0000 7f3aa263e640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 0x7f3a9c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3aa163c640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c4d0 tx=0x7f3a8c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c018070 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c007500 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c004a20 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c0cf420 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.725+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0ce400 con 0x7f3a9c158410 2024-03-04T12:12:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.726+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c074f60 con 0x7f3a9c158410 2024-03-04T12:12:22.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.726+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3a8c01b070 con 0x7f3a9c158410 2024-03-04T12:12:22.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.726+0000 7f3a927fc640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:22.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.727+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a8c0898e0 con 0x7f3a9c158410 2024-03-04T12:12:22.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.727+0000 7f3aa1e3d640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:22.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.727+0000 7f3aa1e3d640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a98036000 tx=0x7f3a9803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:22.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.731+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a9c0cdbc0 con 0x7f3a9c158410 2024-03-04T12:12:22.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.940+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a9c066e10 con 0x7f3a9c158410 2024-03-04T12:12:22.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.940+0000 7f3a927fc640 1 -- 172.21.15.148:0/480694278 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3a8c058390 con 0x7f3a9c158410 2024-03-04T12:12:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:22.942 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T12:12:22.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 msgr2=0x7f3a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a98036000 tx=0x7f3a9803b040 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 msgr2=0x7f3a9c0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c4d0 tx=0x7f3a8c00c990 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 shutdown_connections 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a9c161e70 0x7f3a9c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9c158410 0x7f3a9c0cd680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 --2- 172.21.15.148:0/480694278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a9c06acd0 0x7f3a9c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 >> 172.21.15.148:0/480694278 conn(0x7f3a9c0cfef0 msgr2=0x7f3a9c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 shutdown_connections 2024-03-04T12:12:22.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:22.943+0000 7f3aa40a1640 1 -- 172.21.15.148:0/480694278 wait complete. 2024-03-04T12:12:22.956 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-04T12:12:22.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:23.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.055+0000 7fb7adbb2640 1 Processor -- start 2024-03-04T12:12:23.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 -- start start 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 0x7fb7a80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a80cf200 con 0x7fb7a80cfeb0 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a80cc0c0 con 0x7fb7a815dcd0 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7adbb2640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a80ce680 con 0x7fb7a8161fb0 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7a77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.056+0000 7fb7a77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45270/0 (socket says 172.21.15.148:45270) 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a77fe640 1 -- 172.21.15.148:0/6706306 learned_addr learned my addr 172.21.15.148:0/6706306 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a7fff640 1 --2- 172.21.15.148:0/6706306 >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 0x7fb7a80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a77fe640 1 -- 172.21.15.148:0/6706306 >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 msgr2=0x7fb7a80cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a77fe640 1 --2- 172.21.15.148:0/6706306 >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 0x7fb7a80cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a77fe640 1 -- 172.21.15.148:0/6706306 >> v1:172.21.15.191:6789/0 conn(0x7fb7a80cfeb0 legacy=0x7fb7a81618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a77fe640 1 -- 172.21.15.148:0/6706306 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80ce960 con 0x7fb7a815dcd0 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.057+0000 7fb7a7fff640 1 --2- 172.21.15.148:0/6706306 >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 0x7fb7a80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7a77fe640 1 --2- 172.21.15.148:0/6706306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7a806df90 tx=0x7fb79802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa062d296f1adcf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7a6ffd640 1 -- 172.21.15.148:0/6706306 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798035020 con 0x7fb7a815dcd0 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7a6ffd640 1 -- 172.21.15.148:0/6706306 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798031870 con 0x7fb7a815dcd0 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7a6ffd640 1 -- 172.21.15.148:0/6706306 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798031c40 con 0x7fb7a815dcd0 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 -- 172.21.15.148:0/6706306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 msgr2=0x7fb7a80ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/6706306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7a806df90 tx=0x7fb79802b910 comp rx=0 tx=0).stop 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 -- 172.21.15.148:0/6706306 shutdown_connections 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/6706306 >> v2:172.21.15.191:3300/0 conn(0x7fb7a8161fb0 0x7fb7a80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/6706306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a80ce0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 -- 172.21.15.148:0/6706306 >> 172.21.15.148:0/6706306 conn(0x7fb7a80b3480 msgr2=0x7fb7a80b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 -- 172.21.15.148:0/6706306 shutdown_connections 2024-03-04T12:12:23.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.058+0000 7fb7adbb2640 1 -- 172.21.15.148:0/6706306 wait complete. 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.059+0000 7fb7adbb2640 1 Processor -- start 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 -- start start 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7a80cfeb0 0x7fb7a80b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7a8161fb0 0x7fb7a815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a806bff0 con 0x7fb7a815dcd0 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80cef20 con 0x7fb7a8161fb0 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb7adbb2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a815e0b0 con 0x7fb7a80cfeb0 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.060+0000 7fb79effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45286/0 (socket says 172.21.15.148:45286) 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 -- 172.21.15.148:0/3710512171 learned_addr learned my addr 172.21.15.148:0/3710512171 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7a7fff640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7a8161fb0 0x7fb7a815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7a77fe640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7a80cfeb0 0x7fb7a80b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 -- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7a80cfeb0 msgr2=0x7fb7a80b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7a80cfeb0 0x7fb7a80b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 -- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7a8161fb0 msgr2=0x7fb7a815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7a8161fb0 0x7fb7a815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 -- 172.21.15.148:0/3710512171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80b4b30 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb79effd640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b3f0 tx=0x7fb79000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790015020 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790011830 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a815efa0 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790011c20 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.061+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a815f560 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.062+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a815f7a0 con 0x7fb7a815dcd0 2024-03-04T12:12:23.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.063+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb790018020 con 0x7fb7a815dcd0 2024-03-04T12:12:23.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.063+0000 7fb7a4ff9640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 0x7fb78806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.063+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb79008a180 con 0x7fb7a815dcd0 2024-03-04T12:12:23.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.063+0000 7fb7a77fe640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 0x7fb78806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.064+0000 7fb7a77fe640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 0x7fb78806d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.067+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7a815f7a0 con 0x7fb7a815dcd0 2024-03-04T12:12:23.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.239+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7a806b180 con 0x7fb7a815dcd0 2024-03-04T12:12:23.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.241+0000 7fb7a4ff9640 1 -- 172.21.15.148:0/3710512171 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7fb79001f030 con 0x7fb7a815dcd0 2024-03-04T12:12:23.242 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:23.242 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:23.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 msgr2=0x7fb78806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 0x7fb78806d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 msgr2=0x7fb7a815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b3f0 tx=0x7fb79000b8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 shutdown_connections 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb78806b4d0 0x7fb78806d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7a8161fb0 0x7fb7a815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a815dcd0 0x7fb7a815c220 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 --2- 172.21.15.148:0/3710512171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7a80cfeb0 0x7fb7a80b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 >> 172.21.15.148:0/3710512171 conn(0x7fb7a80b3480 msgr2=0x7fb7a80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 shutdown_connections 2024-03-04T12:12:23.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.243+0000 7fb7adbb2640 1 -- 172.21.15.148:0/3710512171 wait complete. 2024-03-04T12:12:23.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:23.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:23.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.358+0000 7f15bd969640 1 Processor -- start 2024-03-04T12:12:23.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.358+0000 7f15bd969640 1 -- start start 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15bd969640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15b80ccbd0 0x7f15b80b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15bd969640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b815ca50 con 0x7f15b815c2e0 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15bd969640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b815cd30 con 0x7f15b80ceec0 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15bd969640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b815d2f0 con 0x7f15b80ccbd0 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15b6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15b80ccbd0 0x7f15b80b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15b67fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f15b815c2e0 0x7f15b80ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42526/0 (socket says 172.21.15.148:42526) 2024-03-04T12:12:23.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.359+0000 7f15b67fc640 1 -- 172.21.15.148:0/2018393508 learned_addr learned my addr 172.21.15.148:0/2018393508 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:23.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b6ffd640 1 -- 172.21.15.148:0/2018393508 >> v1:172.21.15.191:6789/0 conn(0x7f15b80ceec0 legacy=0x7f15b80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b6ffd640 1 -- 172.21.15.148:0/2018393508 >> v1:172.21.15.148:6789/0 conn(0x7f15b815c2e0 legacy=0x7f15b80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b5ffb640 1 -- 172.21.15.148:0/2018393508 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058821686 0 0) 0x7f15b815ca50 con 0x7f15b815c2e0 2024-03-04T12:12:23.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b6ffd640 1 -- 172.21.15.148:0/2018393508 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b815d010 con 0x7f15b80ccbd0 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b6ffd640 1 --2- 172.21.15.148:0/2018393508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b80b7320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009d90 tx=0x7f15a802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeeaa2c7c15be9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.360+0000 7f15b5ffb640 1 -- 172.21.15.148:0/2018393508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039070 con 0x7f15b80ccbd0 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15b5ffb640 1 -- 172.21.15.148:0/2018393508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a802cd00 con 0x7f15b80ccbd0 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15b5ffb640 1 -- 172.21.15.148:0/2018393508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8034660 con 0x7f15b80ccbd0 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 -- 172.21.15.148:0/2018393508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 msgr2=0x7f15b80b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 --2- 172.21.15.148:0/2018393508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b80b7320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009d90 tx=0x7f15a802bf50 comp rx=0 tx=0).stop 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 -- 172.21.15.148:0/2018393508 shutdown_connections 2024-03-04T12:12:23.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 --2- 172.21.15.148:0/2018393508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b80b7320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 -- 172.21.15.148:0/2018393508 >> 172.21.15.148:0/2018393508 conn(0x7f15b8066dc0 msgr2=0x7f15b80671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 -- 172.21.15.148:0/2018393508 shutdown_connections 2024-03-04T12:12:23.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.361+0000 7f15bd969640 1 -- 172.21.15.148:0/2018393508 wait complete. 2024-03-04T12:12:23.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.362+0000 7f15bd969640 1 Processor -- start 2024-03-04T12:12:23.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.362+0000 7f15bd969640 1 -- start start 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15b80ceec0 0x7f15b80c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15b815c2e0 0x7f15b8158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80ccfb0 con 0x7f15b80ccbd0 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815d6d0 con 0x7f15b80ceec0 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15bd969640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815eb70 con 0x7f15b815c2e0 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45302/0 (socket says 172.21.15.148:45302) 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.363+0000 7f15b6ffd640 1 -- 172.21.15.148:0/3044042476 learned_addr learned my addr 172.21.15.148:0/3044042476 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 -- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15b815c2e0 msgr2=0x7f15b8158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15b815c2e0 0x7f15b8158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 -- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15b80ceec0 msgr2=0x7f15b80c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15b80ceec0 0x7f15b80c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 -- 172.21.15.148:0/3044042476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b81511a0 con 0x7f15b80ccbd0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15b6ffd640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15a8037040 tx=0x7f15a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8044020 con 0x7f15b80ccbd0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a803f670 con 0x7f15b80ccbd0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039040 con 0x7f15b80ccbd0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b80c6790 con 0x7f15b80ccbd0 2024-03-04T12:12:23.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.364+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b81567e0 con 0x7f15b80ccbd0 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.365+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b8067700 con 0x7f15b80ccbd0 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.365+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f15a8047020 con 0x7f15b80ccbd0 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.366+0000 7f159ffff640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.366+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15a80ad1a0 con 0x7f15b80ccbd0 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.366+0000 7f15b67fc640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.366+0000 7f15b67fc640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 0x7f15a406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15ac000c00 tx=0x7f15ac0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.370+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15b8067700 con 0x7f15b80ccbd0 2024-03-04T12:12:23.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.540+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15b8074f60 con 0x7f15b80ccbd0 2024-03-04T12:12:23.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.541+0000 7f159ffff640 1 -- 172.21.15.148:0/3044042476 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15a804e030 con 0x7f15b80ccbd0 2024-03-04T12:12:23.542 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 msgr2=0x7f15a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 0x7f15a406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15ac000c00 tx=0x7f15ac0079c0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 msgr2=0x7f15b8150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15a8037040 tx=0x7f15a8004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 shutdown_connections 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15b815c2e0 0x7f15b8158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15b80ceec0 0x7f15b80c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 --2- 172.21.15.148:0/3044042476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b80ccbd0 0x7f15b8150c60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 >> 172.21.15.148:0/3044042476 conn(0x7f15b8066dc0 msgr2=0x7f15b8159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 shutdown_connections 2024-03-04T12:12:23.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.543+0000 7f15bd969640 1 -- 172.21.15.148:0/3044042476 wait complete. 2024-03-04T12:12:23.555 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:23.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 Processor -- start 2024-03-04T12:12:23.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 -- start start 2024-03-04T12:12:23.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a5015f660 0x7f5a500d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 0x7f5a500cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5015bf50 con 0x7f5a5015bb70 2024-03-04T12:12:23.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a500cf890 con 0x7f5a5015f660 2024-03-04T12:12:23.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.657+0000 7f5a5631b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500b1150 con 0x7f5a500d1cc0 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a5015f660 0x7f5a500d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a5015f660 0x7f5a500d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45316/0 (socket says 172.21.15.148:45316) 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/317880596 learned_addr learned my addr 172.21.15.148:0/317880596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a548b8640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 0x7f5a500cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/317880596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 msgr2=0x7f5a500cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 0x7f5a500cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/317880596 >> v1:172.21.15.191:6789/0 conn(0x7f5a5015bb70 legacy=0x7f5a500b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/317880596 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5015fb60 con 0x7f5a5015f660 2024-03-04T12:12:23.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.658+0000 7f5a548b8640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 0x7f5a500cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a4f7fe640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015f660 0x7f5a500d1780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009870 tx=0x7f5a4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eb0ae2f0840b46 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a4effd640 1 -- 172.21.15.148:0/317880596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44035020 con 0x7f5a5015f660 2024-03-04T12:12:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a4effd640 1 -- 172.21.15.148:0/317880596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a44031640 con 0x7f5a5015f660 2024-03-04T12:12:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a4effd640 1 -- 172.21.15.148:0/317880596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4402d040 con 0x7f5a5015f660 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 -- 172.21.15.148:0/317880596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015f660 msgr2=0x7f5a500d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015f660 0x7f5a500d1780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009870 tx=0x7f5a4402b8f0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 -- 172.21.15.148:0/317880596 shutdown_connections 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a500d1cc0 0x7f5a500cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 --2- 172.21.15.148:0/317880596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015f660 0x7f5a500d1780 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 -- 172.21.15.148:0/317880596 >> 172.21.15.148:0/317880596 conn(0x7f5a500b33a0 msgr2=0x7f5a500b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 -- 172.21.15.148:0/317880596 shutdown_connections 2024-03-04T12:12:23.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.659+0000 7f5a5631b640 1 -- 172.21.15.148:0/317880596 wait complete. 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 Processor -- start 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 -- start start 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 0x7f5a501f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 0x7f5a5015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50160400 con 0x7f5a5015bb70 2024-03-04T12:12:23.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50160120 con 0x7f5a500d1cc0 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.661+0000 7f5a5631b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500d0260 con 0x7f5a5015f660 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45322/0 (socket says 172.21.15.148:45322) 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/3492876277 learned_addr learned my addr 172.21.15.148:0/3492876277 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a548b8640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 0x7f5a5015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4ffff640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 0x7f5a501f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 msgr2=0x7f5a5015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 0x7f5a5015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 msgr2=0x7f5a501f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 0x7f5a501f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 -- 172.21.15.148:0/3492876277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5015d230 con 0x7f5a5015bb70 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4ffff640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 0x7f5a501f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a548b8640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 0x7f5a5015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4f7fe640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44040070 con 0x7f5a5015bb70 2024-03-04T12:12:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a44039bf0 con 0x7f5a5015bb70 2024-03-04T12:12:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5015d4d0 con 0x7f5a5015bb70 2024-03-04T12:12:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44035040 con 0x7f5a5015bb70 2024-03-04T12:12:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.662+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5015da90 con 0x7f5a5015bb70 2024-03-04T12:12:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.663+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50074f60 con 0x7f5a5015bb70 2024-03-04T12:12:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.664+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5a4404b060 con 0x7f5a5015bb70 2024-03-04T12:12:23.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.664+0000 7f5a4cff9640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 0x7f5a3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.664+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a440abd90 con 0x7f5a5015bb70 2024-03-04T12:12:23.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.664+0000 7f5a4ffff640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 0x7f5a3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.664+0000 7f5a4ffff640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 0x7f5a3c06d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a500b4b20 tx=0x7f5a40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.668+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a4407a960 con 0x7f5a5015bb70 2024-03-04T12:12:23.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.838+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a500b4e90 con 0x7f5a5015bb70 2024-03-04T12:12:23.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.838+0000 7f5a4cff9640 1 -- 172.21.15.148:0/3492876277 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a4407a260 con 0x7f5a5015bb70 2024-03-04T12:12:23.840 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 msgr2=0x7f5a3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 0x7f5a3c06d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a500b4b20 tx=0x7f5a40009040 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 msgr2=0x7f5a5006c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a440046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 shutdown_connections 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5a3c06b500 0x7f5a3c06d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a5015f660 0x7f5a5015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a5015bb70 0x7f5a5006c840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 --2- 172.21.15.148:0/3492876277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a500d1cc0 0x7f5a501f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 >> 172.21.15.148:0/3492876277 conn(0x7f5a500b33a0 msgr2=0x7f5a500b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 shutdown_connections 2024-03-04T12:12:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.840+0000 7f5a5631b640 1 -- 172.21.15.148:0/3492876277 wait complete. 2024-03-04T12:12:23.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:23.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.955+0000 7f52ce337640 1 Processor -- start 2024-03-04T12:12:23.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.955+0000 7f52ce337640 1 -- start start 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52ce337640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f52c80b48a0 0x7f52c80b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52ce337640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c806d8a0 con 0x7f52c806d3c0 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52ce337640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c806db80 con 0x7f52c815a810 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52ce337640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c806de60 con 0x7f52c80b48a0 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52cc8d4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f52c806d3c0 0x7f52c80b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42530/0 (socket says 172.21.15.148:42530) 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52cc8d4640 1 -- 172.21.15.148:0/988736463 learned_addr learned my addr 172.21.15.148:0/988736463 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c77fe640 1 --2- 172.21.15.148:0/988736463 >> v2:172.21.15.191:3300/0 conn(0x7f52c80b48a0 0x7f52c80b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446887609 0 0) 0x7f52c806d8a0 con 0x7f52c806d3c0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b0003520 con 0x7f52c806d3c0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114984550 0 0) 0x7f52c806db80 con 0x7f52c815a810 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c806d8a0 con 0x7f52c815a810 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2822532706 0 0) 0x7f52b0003520 con 0x7f52c806d3c0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 >> v2:172.21.15.191:3300/0 conn(0x7f52c80b48a0 msgr2=0x7f52c80b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 --2- 172.21.15.148:0/988736463 >> v2:172.21.15.191:3300/0 conn(0x7f52c80b48a0 0x7f52c80b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 >> v1:172.21.15.191:6789/0 conn(0x7f52c815a810 legacy=0x7f52c80b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c806e140 con 0x7f52c806d3c0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f52c00019f0 con 0x7f52c806d3c0 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c77fe640 1 --2- 172.21.15.148:0/988736463 >> v2:172.21.15.191:3300/0 conn(0x7f52c80b48a0 0x7f52c80b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 >> v1:172.21.15.148:6789/0 conn(0x7f52c806d3c0 legacy=0x7f52c80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b0003b40 0x7f52b0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b0004800 0x7f52b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c806de60 con 0x7f52b00051c0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b000c040 con 0x7f52b0003b40 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.956+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b000c240 con 0x7f52b0004800 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52c7fff640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b0003b40 0x7f52b0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52c77fe640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b0004800 0x7f52b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 -- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b0004800 msgr2=0x7f52b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b0004800 0x7f52b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 -- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b0003b40 msgr2=0x7f52b0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b0003b40 0x7f52b0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 -- 172.21.15.148:0/988736463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0003520 con 0x7f52b00051c0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.957+0000 7f52cc8d4640 1 -- 172.21.15.148:0/988736463 reap_dead start 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52cc8d4640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52b000b9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52c806d7d0 tx=0x7f52b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be9e3cfc145749 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000a410 con 0x7f52b00051c0 2024-03-04T12:12:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c0034650 con 0x7f52b00051c0 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52c6ffd640 1 -- 172.21.15.148:0/988736463 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0034a40 con 0x7f52b00051c0 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 -- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 msgr2=0x7f52b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52b000b9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52c806d7d0 tx=0x7f52b0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 -- 172.21.15.148:0/988736463 shutdown_connections 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 --2- 172.21.15.148:0/988736463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52b000b9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 -- 172.21.15.148:0/988736463 >> 172.21.15.148:0/988736463 conn(0x7f52c806acd0 msgr2=0x7f52c806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 -- 172.21.15.148:0/988736463 shutdown_connections 2024-03-04T12:12:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.958+0000 7f52ce337640 1 -- 172.21.15.148:0/988736463 wait complete. 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.959+0000 7f52ce337640 1 Processor -- start 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 -- start start 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c8156260 0x7f52c81f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c80ac400 0x7f52c80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c815e3c0 con 0x7f52b00051c0 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80cc1c0 con 0x7f52c8156260 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52ce337640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c815e540 con 0x7f52c80ac400 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45344/0 (socket says 172.21.15.148:45344) 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.960+0000 7f52c7fff640 1 -- 172.21.15.148:0/336802245 learned_addr learned my addr 172.21.15.148:0/336802245 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52cc8d4640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c80ac400 0x7f52c80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c77fe640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c8156260 0x7f52c81f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 -- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c80ac400 msgr2=0x7f52c80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c80ac400 0x7f52c80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 -- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c8156260 msgr2=0x7f52c81f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c8156260 0x7f52c81f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 -- 172.21.15.148:0/336802245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c81f1290 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c7fff640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52b800f7d0 tx=0x7f52b800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b801a070 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b800fe80 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8007050 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c80aea60 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.961+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c81f1960 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.962+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c8074f60 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.963+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f52b801d070 con 0x7f52b00051c0 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.963+0000 7f52c4ff9640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.963+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f52b8089d10 con 0x7f52b00051c0 2024-03-04T12:12:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.963+0000 7f52c77fe640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.963+0000 7f52c77fe640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 0x7f52b406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52c80affb0 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:23.967+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52b8058840 con 0x7f52b00051c0 2024-03-04T12:12:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.138+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52c806ec30 con 0x7f52b00051c0 2024-03-04T12:12:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.139+0000 7f52c4ff9640 1 -- 172.21.15.148:0/336802245 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52b805e020 con 0x7f52b00051c0 2024-03-04T12:12:24.140 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 msgr2=0x7f52b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 0x7f52b406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52c80affb0 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 msgr2=0x7f52c80ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52b800f7d0 tx=0x7f52b800fc90 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 shutdown_connections 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c80ac400 0x7f52c80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c8156260 0x7f52c81f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 --2- 172.21.15.148:0/336802245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52b00051c0 0x7f52c80ae3f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 >> 172.21.15.148:0/336802245 conn(0x7f52c806acd0 msgr2=0x7f52c80b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 shutdown_connections 2024-03-04T12:12:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.141+0000 7f52ce337640 1 -- 172.21.15.148:0/336802245 wait complete. 2024-03-04T12:12:24.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.256+0000 7f1906376640 1 Processor -- start 2024-03-04T12:12:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.256+0000 7f1906376640 1 -- start start 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1906376640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 0x7f19000b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1906376640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19000cf020 0x7f190015d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1906376640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19000ca970 con 0x7f19000d3300 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1906376640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1900161d90 con 0x7f19000cf020 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1906376640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900162070 con 0x7f19000cec40 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18ff7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19000cf020 0x7f190015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1904913640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f19000d3300 0x7f19000d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42546/0 (socket says 172.21.15.148:42546) 2024-03-04T12:12:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f1904913640 1 -- 172.21.15.148:0/3199848977 learned_addr learned my addr 172.21.15.148:0/3199848977 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18fffff640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 0x7f19000b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18ff7fe640 1 -- 172.21.15.148:0/3199848977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 msgr2=0x7f19000b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 0x7f19000b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18ff7fe640 1 -- 172.21.15.148:0/3199848977 >> v1:172.21.15.148:6789/0 conn(0x7f19000d3300 legacy=0x7f19000d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18feffd640 1 -- 172.21.15.148:0/3199848977 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397729602 0 0) 0x7f19000ca970 con 0x7f19000d3300 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18ff7fe640 1 -- 172.21.15.148:0/3199848977 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900162350 con 0x7f19000cf020 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.257+0000 7f18fffff640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 0x7f19000b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cf020 0x7f190015d730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097a0 tx=0x7f18f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2aad47d9056437 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f18feffd640 1 -- 172.21.15.148:0/3199848977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4035020 con 0x7f19000cf020 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f18feffd640 1 -- 172.21.15.148:0/3199848977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f4031830 con 0x7f19000cf020 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f18feffd640 1 -- 172.21.15.148:0/3199848977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4031c00 con 0x7f19000cf020 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f1906376640 1 -- 172.21.15.148:0/3199848977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cf020 msgr2=0x7f190015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.258+0000 7f1906376640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cf020 0x7f190015d730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097a0 tx=0x7f18f402b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 -- 172.21.15.148:0/3199848977 shutdown_connections 2024-03-04T12:12:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cf020 0x7f190015d730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 --2- 172.21.15.148:0/3199848977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000cec40 0x7f19000b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 -- 172.21.15.148:0/3199848977 >> 172.21.15.148:0/3199848977 conn(0x7f19000d0580 msgr2=0x7f19000d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 -- 172.21.15.148:0/3199848977 shutdown_connections 2024-03-04T12:12:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.259+0000 7f1906376640 1 -- 172.21.15.148:0/3199848977 wait complete. 2024-03-04T12:12:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.260+0000 7f1906376640 1 Processor -- start 2024-03-04T12:12:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.260+0000 7f1906376640 1 -- start start 2024-03-04T12:12:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.260+0000 7f1906376640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.260+0000 7f1906376640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 0x7f190015d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 0x7f1900149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001617d0 con 0x7f19000cec40 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000d1bf0 con 0x7f19000cf020 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900161950 con 0x7f19000d3300 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45358/0 (socket says 172.21.15.148:45358) 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 -- 172.21.15.148:0/1948739702 learned_addr learned my addr 172.21.15.148:0/1948739702 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 0x7f190015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 -- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 msgr2=0x7f1900149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1904913640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 0x7f1900149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 0x7f1900149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 -- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 msgr2=0x7f190015d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 0x7f190015d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 -- 172.21.15.148:0/1948739702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190015dde0 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1904913640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 0x7f1900149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 0x7f190015d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fffff640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7f0 tx=0x7f18f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0015070 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0007a30 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190015e0c0 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007e00 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.261+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190015e680 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.262+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1900074f60 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.263+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f18f0018070 con 0x7f19000cec40 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.263+0000 7f18fcff9640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.263+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.263+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f18f008a4f0 con 0x7f19000cec40 2024-03-04T12:12:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.264+0000 7f18ff7fe640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 0x7f18ec06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f18f4036000 tx=0x7f18f4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.268+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18f0059030 con 0x7f19000cec40 2024-03-04T12:12:24.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:12:24.288 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:12:24.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.446+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19000d1930 con 0x7f19000cec40 2024-03-04T12:12:24.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.446+0000 7f18fcff9640 1 -- 172.21.15.148:0/1948739702 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f18f0058930 con 0x7f19000cec40 2024-03-04T12:12:24.448 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 msgr2=0x7f18ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 0x7f18ec06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f18f4036000 tx=0x7f18f4008030 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 msgr2=0x7f19000c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7f0 tx=0x7f18f000cbf0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 shutdown_connections 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19000d3300 0x7f1900149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19000cf020 0x7f190015d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 --2- 172.21.15.148:0/1948739702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000cec40 0x7f19000c68b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 >> 172.21.15.148:0/1948739702 conn(0x7f19000d0580 msgr2=0x7f190015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 shutdown_connections 2024-03-04T12:12:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.448+0000 7f1906376640 1 -- 172.21.15.148:0/1948739702 wait complete. 2024-03-04T12:12:24.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:24.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.560+0000 7f0e36c8e640 1 Processor -- start 2024-03-04T12:12:24.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.560+0000 7f0e36c8e640 1 -- start start 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e36c8e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e300cfae0 0x7f0e300714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e36c8e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3015f020 con 0x7f0e300cf650 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e36c8e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e300cd010 con 0x7f0e30154a80 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e36c8e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e300cff70 con 0x7f0e300cfae0 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e3522b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e300cfae0 0x7f0e300714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e2ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0e300cf650 0x7f0e30070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42552/0 (socket says 172.21.15.148:42552) 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e2ffff640 1 -- 172.21.15.148:0/2336371376 learned_addr learned my addr 172.21.15.148:0/2336371376 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e3522b640 1 -- 172.21.15.148:0/2336371376 >> v1:172.21.15.191:6789/0 conn(0x7f0e30154a80 legacy=0x7f0e3015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e3522b640 1 -- 172.21.15.148:0/2336371376 >> v1:172.21.15.148:6789/0 conn(0x7f0e300cf650 legacy=0x7f0e30070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e2f7fe640 1 -- 172.21.15.148:0/2336371376 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311509594 0 0) 0x7f0e24001110 con 0x7f0e300cf650 2024-03-04T12:12:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.561+0000 7f0e3522b640 1 -- 172.21.15.148:0/2336371376 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e300cdaa0 con 0x7f0e300cfae0 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.562+0000 7f0e3522b640 1 --2- 172.21.15.148:0/2336371376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e300714f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009d90 tx=0x7f0e2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23026159568155ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.562+0000 7f0e2f7fe640 1 -- 172.21.15.148:0/2336371376 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28039070 con 0x7f0e300cfae0 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.562+0000 7f0e2f7fe640 1 -- 172.21.15.148:0/2336371376 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2802cc60 con 0x7f0e300cfae0 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.562+0000 7f0e2f7fe640 1 -- 172.21.15.148:0/2336371376 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28034660 con 0x7f0e300cfae0 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 -- 172.21.15.148:0/2336371376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 msgr2=0x7f0e300714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/2336371376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e300714f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009d90 tx=0x7f0e2802beb0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 -- 172.21.15.148:0/2336371376 shutdown_connections 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/2336371376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e300714f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 -- 172.21.15.148:0/2336371376 >> 172.21.15.148:0/2336371376 conn(0x7f0e3006f5f0 msgr2=0x7f0e3006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 -- 172.21.15.148:0/2336371376 shutdown_connections 2024-03-04T12:12:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.563+0000 7f0e36c8e640 1 -- 172.21.15.148:0/2336371376 wait complete. 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.564+0000 7f0e36c8e640 1 Processor -- start 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.564+0000 7f0e36c8e640 1 -- start start 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e300cf650 0x7f0e300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30154a80 0x7f0e301613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300ce340 con 0x7f0e300cfae0 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300ce060 con 0x7f0e30154a80 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e36c8e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300ce620 con 0x7f0e300cf650 2024-03-04T12:12:24.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45378/0 (socket says 172.21.15.148:45378) 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 -- 172.21.15.148:0/1661464934 learned_addr learned my addr 172.21.15.148:0/1661464934 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e34a2a640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e300cf650 0x7f0e300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e3522b640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30154a80 0x7f0e301613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 -- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e300cf650 msgr2=0x7f0e300b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e300cf650 0x7f0e300b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 -- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30154a80 msgr2=0x7f0e301613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30154a80 0x7f0e301613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.565+0000 7f0e2ffff640 1 -- 172.21.15.148:0/1661464934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e300c66d0 con 0x7f0e300cfae0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e2ffff640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c7b0 tx=0x7f0e2400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24018070 con 0x7f0e300cfae0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240072d0 con 0x7f0e300cfae0 2024-03-04T12:12:24.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e301f41b0 con 0x7f0e300cfae0 2024-03-04T12:12:24.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240047e0 con 0x7f0e300cfae0 2024-03-04T12:12:24.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.566+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e300b6470 con 0x7f0e300cfae0 2024-03-04T12:12:24.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.567+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e300b3150 con 0x7f0e300cfae0 2024-03-04T12:12:24.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.567+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e2401b070 con 0x7f0e300cfae0 2024-03-04T12:12:24.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.567+0000 7f0e2d7fa640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 0x7f0e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.567+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0e24089820 con 0x7f0e300cfae0 2024-03-04T12:12:24.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.567+0000 7f0e34a2a640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 0x7f0e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.568+0000 7f0e34a2a640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 0x7f0e1c06d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006fd0 tx=0x7f0e20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.572+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e300b3150 con 0x7f0e300cfae0 2024-03-04T12:12:24.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.776+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e30074f60 con 0x7f0e300cfae0 2024-03-04T12:12:24.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.776+0000 7f0e2d7fa640 1 -- 172.21.15.148:0/1661464934 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f0e300c66d0 con 0x7f0e300cfae0 2024-03-04T12:12:24.779 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:24.780 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:24.780 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.778+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 msgr2=0x7f0e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.778+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 0x7f0e1c06d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006fd0 tx=0x7f0e20008040 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.778+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 msgr2=0x7f0e301f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.778+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c7b0 tx=0x7f0e2400cc70 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 shutdown_connections 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e1c06b4d0 0x7f0e1c06d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30154a80 0x7f0e301613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e300cfae0 0x7f0e301f3c70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 --2- 172.21.15.148:0/1661464934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e300cf650 0x7f0e300b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 >> 172.21.15.148:0/1661464934 conn(0x7f0e3006f5f0 msgr2=0x7f0e3006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 shutdown_connections 2024-03-04T12:12:24.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.779+0000 7f0e36c8e640 1 -- 172.21.15.148:0/1661464934 wait complete. 2024-03-04T12:12:24.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:24.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.889+0000 7f1990238640 1 Processor -- start 2024-03-04T12:12:24.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.889+0000 7f1990238640 1 -- start start 2024-03-04T12:12:24.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f1990238640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f198815b030 0x7f198815c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f1990238640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f198815b410 0x7f198815b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f1990238640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198806d8b0 con 0x7f198806d3b0 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f1990238640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880b2720 con 0x7f198815b030 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f1990238640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198806db90 con 0x7f198815b410 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f198dfd4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f198815b030 0x7f198815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f198dfd4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f198815b030 0x7f198815c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45388/0 (socket says 172.21.15.148:45388) 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.890+0000 7f198dfd4640 1 -- 172.21.15.148:0/2614067553 learned_addr learned my addr 172.21.15.148:0/2614067553 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.891+0000 7f198e7d5640 1 --2- 172.21.15.148:0/2614067553 >> v2:172.21.15.191:3300/0 conn(0x7f198815b410 0x7f198815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.891+0000 7f198dfd4640 1 -- 172.21.15.148:0/2614067553 >> v2:172.21.15.191:3300/0 conn(0x7f198815b410 msgr2=0x7f198815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.891+0000 7f198dfd4640 1 --2- 172.21.15.148:0/2614067553 >> v2:172.21.15.191:3300/0 conn(0x7f198815b410 0x7f198815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.891+0000 7f198dfd4640 1 -- 172.21.15.148:0/2614067553 >> v1:172.21.15.191:6789/0 conn(0x7f198806d3b0 legacy=0x7f198815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:24.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.891+0000 7f198dfd4640 1 -- 172.21.15.148:0/2614067553 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806de70 con 0x7f198815b030 2024-03-04T12:12:24.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f198dfd4640 1 --2- 172.21.15.148:0/2614067553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198815b030 0x7f198815c120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f19880b28a0 tx=0x7f197802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c1c2fe8ece6a5d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f198cfd2640 1 -- 172.21.15.148:0/2614067553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197803b070 con 0x7f198815b030 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f198cfd2640 1 -- 172.21.15.148:0/2614067553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197802ee00 con 0x7f198815b030 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f198cfd2640 1 -- 172.21.15.148:0/2614067553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978036660 con 0x7f198815b030 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 -- 172.21.15.148:0/2614067553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198815b030 msgr2=0x7f198815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 --2- 172.21.15.148:0/2614067553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198815b030 0x7f198815c120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f19880b28a0 tx=0x7f197802e050 comp rx=0 tx=0).stop 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 -- 172.21.15.148:0/2614067553 shutdown_connections 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 --2- 172.21.15.148:0/2614067553 >> v2:172.21.15.191:3300/0 conn(0x7f198815b410 0x7f198815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 --2- 172.21.15.148:0/2614067553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198815b030 0x7f198815c120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 -- 172.21.15.148:0/2614067553 >> 172.21.15.148:0/2614067553 conn(0x7f198806acd0 msgr2=0x7f198806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:24.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 -- 172.21.15.148:0/2614067553 shutdown_connections 2024-03-04T12:12:24.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.892+0000 7f1990238640 1 -- 172.21.15.148:0/2614067553 wait complete. 2024-03-04T12:12:24.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.893+0000 7f1990238640 1 Processor -- start 2024-03-04T12:12:24.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 -- start start 2024-03-04T12:12:24.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198815b030 0x7f1988151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f198815b410 0x7f1988151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198806e690 con 0x7f198806d3b0 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815bdc0 con 0x7f198815b410 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f1990238640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198806e810 con 0x7f198815b030 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f198dfd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f198dfd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45396/0 (socket says 172.21.15.148:45396) 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.894+0000 7f198dfd4640 1 -- 172.21.15.148:0/2866697058 learned_addr learned my addr 172.21.15.148:0/2866697058 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198e7d5640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f198815b410 0x7f1988151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198d7d3640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198815b030 0x7f1988151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 -- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198815b030 msgr2=0x7f1988151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198815b030 0x7f1988151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 -- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f198815b410 msgr2=0x7f1988151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f198815b410 0x7f1988151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 -- 172.21.15.148:0/2866697058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198813c360 con 0x7f198806d3b0 2024-03-04T12:12:24.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f198dfd4640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f197802e090 tx=0x7f19780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978045070 con 0x7f198806d3b0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978040670 con 0x7f198806d3b0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988152530 con 0x7f198806d3b0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197803b040 con 0x7f198806d3b0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.895+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988152af0 con 0x7f198806d3b0 2024-03-04T12:12:24.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.896+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1988074f60 con 0x7f198806d3b0 2024-03-04T12:12:24.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.897+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f197803e070 con 0x7f198806d3b0 2024-03-04T12:12:24.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.897+0000 7f197e7fc640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 0x7f196c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:24.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.897+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f19780ae840 con 0x7f198806d3b0 2024-03-04T12:12:24.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.897+0000 7f198d7d3640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 0x7f196c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:24.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.897+0000 7f198d7d3640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 0x7f196c06d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19880cdc00 tx=0x7f1984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:24.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:24.901+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f197807d410 con 0x7f198806d3b0 2024-03-04T12:12:25.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.078+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198813c530 con 0x7f198806d3b0 2024-03-04T12:12:25.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.079+0000 7f197e7fc640 1 -- 172.21.15.148:0/2866697058 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f197807cd10 con 0x7f198806d3b0 2024-03-04T12:12:25.081 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:25.081 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 msgr2=0x7f196c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 0x7f196c06d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19880cdc00 tx=0x7f1984009040 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 msgr2=0x7f198813ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f197802e090 tx=0x7f19780046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 shutdown_connections 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f196c06b500 0x7f196c06d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f198815b410 0x7f1988151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198815b030 0x7f1988151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 --2- 172.21.15.148:0/2866697058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f198806d3b0 0x7f198813ba80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 >> 172.21.15.148:0/2866697058 conn(0x7f198806acd0 msgr2=0x7f198815d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 shutdown_connections 2024-03-04T12:12:25.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.081+0000 7f1990238640 1 -- 172.21.15.148:0/2866697058 wait complete. 2024-03-04T12:12:25.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:25.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:25.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.196+0000 7fba87806640 1 Processor -- start 2024-03-04T12:12:25.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.196+0000 7fba87806640 1 -- start start 2024-03-04T12:12:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fba8015a810 0x7fba800b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba800cc000 0x7fba800b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015e3c0 0x7fba8015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8006f770 con 0x7fba800cc000 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8006d710 con 0x7fba8015a810 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba87806640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006d9f0 con 0x7fba8015e3c0 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba84da1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba800cc000 0x7fba800b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba84da1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba800cc000 0x7fba800b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45404/0 (socket says 172.21.15.148:45404) 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.197+0000 7fba84da1640 1 -- 172.21.15.148:0/3722308758 learned_addr learned my addr 172.21.15.148:0/3722308758 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba855a2640 1 --2- 172.21.15.148:0/3722308758 >> v2:172.21.15.191:3300/0 conn(0x7fba8015a810 0x7fba800b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba85da3640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015e3c0 0x7fba8015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 -- 172.21.15.148:0/3722308758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015e3c0 msgr2=0x7fba8015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015e3c0 0x7fba8015abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 -- 172.21.15.148:0/3722308758 >> v2:172.21.15.191:3300/0 conn(0x7fba8015a810 msgr2=0x7fba800b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 --2- 172.21.15.148:0/3722308758 >> v2:172.21.15.191:3300/0 conn(0x7fba8015a810 0x7fba800b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 -- 172.21.15.148:0/3722308758 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8006dcd0 con 0x7fba800cc000 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.198+0000 7fba84da1640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800b46d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fba800d0660 tx=0x7fba7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a644a57ab0883ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba73fff640 1 -- 172.21.15.148:0/3722308758 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c035020 con 0x7fba800cc000 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba73fff640 1 -- 172.21.15.148:0/3722308758 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c031870 con 0x7fba800cc000 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba73fff640 1 -- 172.21.15.148:0/3722308758 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c031c40 con 0x7fba800cc000 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 -- 172.21.15.148:0/3722308758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 msgr2=0x7fba800b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800b46d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fba800d0660 tx=0x7fba7c02b910 comp rx=0 tx=0).stop 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 -- 172.21.15.148:0/3722308758 shutdown_connections 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015e3c0 0x7fba8015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 --2- 172.21.15.148:0/3722308758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800b46d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 --2- 172.21.15.148:0/3722308758 >> v2:172.21.15.191:3300/0 conn(0x7fba8015a810 0x7fba800b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 -- 172.21.15.148:0/3722308758 >> 172.21.15.148:0/3722308758 conn(0x7fba8006acd0 msgr2=0x7fba8006b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 -- 172.21.15.148:0/3722308758 shutdown_connections 2024-03-04T12:12:25.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.199+0000 7fba87806640 1 -- 172.21.15.148:0/3722308758 wait complete. 2024-03-04T12:12:25.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.200+0000 7fba87806640 1 Processor -- start 2024-03-04T12:12:25.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.200+0000 7fba87806640 1 -- start start 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015a810 0x7fba80159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba8015e3c0 0x7fba800b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006d3c0 con 0x7fba800cc000 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006e290 con 0x7fba8015e3c0 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba87806640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8015b130 con 0x7fba8015a810 2024-03-04T12:12:25.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba855a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba855a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45414/0 (socket says 172.21.15.148:45414) 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba855a2640 1 -- 172.21.15.148:0/2857699662 learned_addr learned my addr 172.21.15.148:0/2857699662 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba84da1640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015a810 0x7fba80159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.201+0000 7fba85da3640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba8015e3c0 0x7fba800b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 -- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015a810 msgr2=0x7fba80159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015a810 0x7fba80159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 -- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba8015e3c0 msgr2=0x7fba800b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba8015e3c0 0x7fba800b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 -- 172.21.15.148:0/2857699662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80158520 con 0x7fba800cc000 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba855a2640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d800 tx=0x7fba7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74018020 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba740096a0 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80158800 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74009a70 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.202+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80158dc0 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.203+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba80074f60 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.204+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fba7401b020 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.204+0000 7fba71ffb640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.204+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba7408a7b0 con 0x7fba800cc000 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.204+0000 7fba84da1640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.204+0000 7fba84da1640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba7c036000 tx=0x7fba7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.208+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba74022030 con 0x7fba800cc000 2024-03-04T12:12:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.379+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba80159e90 con 0x7fba800cc000 2024-03-04T12:12:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.380+0000 7fba71ffb640 1 -- 172.21.15.148:0/2857699662 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba740592f0 con 0x7fba800cc000 2024-03-04T12:12:25.381 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:25.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba7c036000 tx=0x7fba7c008030 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 msgr2=0x7fba800cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d800 tx=0x7fba7400dcc0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 shutdown_connections 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba8015e3c0 0x7fba800b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba8015a810 0x7fba80159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 --2- 172.21.15.148:0/2857699662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba800cc000 0x7fba800cf690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 >> 172.21.15.148:0/2857699662 conn(0x7fba8006acd0 msgr2=0x7fba800b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 shutdown_connections 2024-03-04T12:12:25.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.382+0000 7fba87806640 1 -- 172.21.15.148:0/2857699662 wait complete. 2024-03-04T12:12:25.396 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.498+0000 7f4321b69640 1 Processor -- start 2024-03-04T12:12:25.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.498+0000 7f4321b69640 1 -- start start 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f4321b69640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f431c0cec40 0x7f431c0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f4321b69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 0x7f431c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f4321b69640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c0ca970 con 0x7f431c0d3300 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f4321b69640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c161d90 con 0x7f431c0cec40 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f4321b69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c162070 con 0x7f431c0cf020 2024-03-04T12:12:25.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f431c0cec40 0x7f431c0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f431c0cec40 0x7f431c0b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45428/0 (socket says 172.21.15.148:45428) 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 -- 172.21.15.148:0/3726976450 learned_addr learned my addr 172.21.15.148:0/3726976450 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 -- 172.21.15.148:0/3726976450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 msgr2=0x7f431c15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431affd640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 0x7f431c15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 0x7f431c15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 -- 172.21.15.148:0/3726976450 >> v1:172.21.15.191:6789/0 conn(0x7f431c0d3300 legacy=0x7f431c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431b7fe640 1 -- 172.21.15.148:0/3726976450 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c162350 con 0x7f431c0cec40 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.499+0000 7f431affd640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 0x7f431c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.500+0000 7f431b7fe640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c0b6ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f430c009870 tx=0x7f430c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230946ebdff9a9cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.500+0000 7f431a7fc640 1 -- 172.21.15.148:0/3726976450 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c035020 con 0x7f431c0cec40 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.500+0000 7f431a7fc640 1 -- 172.21.15.148:0/3726976450 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c031640 con 0x7f431c0cec40 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.500+0000 7f431a7fc640 1 -- 172.21.15.148:0/3726976450 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c02d040 con 0x7f431c0cec40 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 -- 172.21.15.148:0/3726976450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 msgr2=0x7f431c0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c0b6ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f430c009870 tx=0x7f430c02b8f0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 -- 172.21.15.148:0/3726976450 shutdown_connections 2024-03-04T12:12:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0cf020 0x7f431c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 --2- 172.21.15.148:0/3726976450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c0b6ea0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 -- 172.21.15.148:0/3726976450 >> 172.21.15.148:0/3726976450 conn(0x7f431c0d0580 msgr2=0x7f431c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 -- 172.21.15.148:0/3726976450 shutdown_connections 2024-03-04T12:12:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.501+0000 7f4321b69640 1 -- 172.21.15.148:0/3726976450 wait complete. 2024-03-04T12:12:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.502+0000 7f4321b69640 1 Processor -- start 2024-03-04T12:12:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.502+0000 7f4321b69640 1 -- start start 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f431c0cf020 0x7f431c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0d3300 0x7f431c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1617d0 con 0x7f431c0cec40 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0d1bf0 con 0x7f431c0cf020 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4321b69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c161950 con 0x7f431c0d3300 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45430/0 (socket says 172.21.15.148:45430) 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 -- 172.21.15.148:0/3368138945 learned_addr learned my addr 172.21.15.148:0/3368138945 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431affd640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f431c0cf020 0x7f431c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431bfff640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0d3300 0x7f431c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 -- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0d3300 msgr2=0x7f431c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0d3300 0x7f431c15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 -- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f431c0cf020 msgr2=0x7f431c15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f431c0cf020 0x7f431c15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 -- 172.21.15.148:0/3368138945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15b0e0 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f431b7fe640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f430c02b930 tx=0x7f430c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.503+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c040070 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.504+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c039c10 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.504+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c15d5a0 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.504+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c035040 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.504+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c15db60 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.504+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c074f60 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.505+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f430c04b060 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.505+0000 7f4303fff640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 0x7f430806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.505+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f430c0abf40 con 0x7f431c0cec40 2024-03-04T12:12:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.505+0000 7f431affd640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 0x7f430806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.506+0000 7f431affd640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 0x7f430806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f431c160430 tx=0x7f4310008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.510+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f430c07a430 con 0x7f431c0cec40 2024-03-04T12:12:25.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.679+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f431c160460 con 0x7f431c0cec40 2024-03-04T12:12:25.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.679+0000 7f4303fff640 1 -- 172.21.15.148:0/3368138945 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f430c07e4d0 con 0x7f431c0cec40 2024-03-04T12:12:25.681 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 msgr2=0x7f430806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 0x7f430806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f431c160430 tx=0x7f4310008040 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 msgr2=0x7f431c1f3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f430c02b930 tx=0x7f430c0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 shutdown_connections 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f430806b4d0 0x7f430806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f431c0d3300 0x7f431c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f431c0cf020 0x7f431c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 --2- 172.21.15.148:0/3368138945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f431c0cec40 0x7f431c1f3ec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 >> 172.21.15.148:0/3368138945 conn(0x7f431c0d0580 msgr2=0x7f431c0c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 shutdown_connections 2024-03-04T12:12:25.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.681+0000 7f4321b69640 1 -- 172.21.15.148:0/3368138945 wait complete. 2024-03-04T12:12:25.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:25.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.794+0000 7fc7f9236640 1 Processor -- start 2024-03-04T12:12:25.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 -- start start 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7f40cf2d0 0x7fc7f40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7f415bca0 0x7fc7f40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f406c680 con 0x7fc7f40af420 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f406b1e0 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.795+0000 7fc7f9236640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40cfbf0 con 0x7fc7f415bca0 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7f40cf2d0 0x7fc7f40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7f40cf2d0 0x7fc7f40cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45440/0 (socket says 172.21.15.148:45440) 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 -- 172.21.15.148:0/2170806064 learned_addr learned my addr 172.21.15.148:0/2170806064 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f259c640 1 --2- 172.21.15.148:0/2170806064 >> v2:172.21.15.191:3300/0 conn(0x7fc7f415bca0 0x7fc7f40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 -- 172.21.15.148:0/2170806064 >> v2:172.21.15.191:3300/0 conn(0x7fc7f415bca0 msgr2=0x7fc7f40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 --2- 172.21.15.148:0/2170806064 >> v2:172.21.15.191:3300/0 conn(0x7fc7f415bca0 0x7fc7f40cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 -- 172.21.15.148:0/2170806064 >> v1:172.21.15.191:6789/0 conn(0x7fc7f40af420 legacy=0x7fc7f40cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.796+0000 7fc7f2d9d640 1 -- 172.21.15.148:0/2170806064 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4066bd0 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f2d9d640 1 --2- 172.21.15.148:0/2170806064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40cf2d0 0x7fc7f40cd490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40712f0 tx=0x7fc7e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbd36f5d7d329cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f1d9b640 1 -- 172.21.15.148:0/2170806064 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4035020 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f1d9b640 1 -- 172.21.15.148:0/2170806064 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4031870 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f1d9b640 1 -- 172.21.15.148:0/2170806064 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4031c40 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 -- 172.21.15.148:0/2170806064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40cf2d0 msgr2=0x7fc7f40cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 --2- 172.21.15.148:0/2170806064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40cf2d0 0x7fc7f40cd490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40712f0 tx=0x7fc7e402b910 comp rx=0 tx=0).stop 2024-03-04T12:12:25.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 -- 172.21.15.148:0/2170806064 shutdown_connections 2024-03-04T12:12:25.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 --2- 172.21.15.148:0/2170806064 >> v2:172.21.15.191:3300/0 conn(0x7fc7f415bca0 0x7fc7f40cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 --2- 172.21.15.148:0/2170806064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40cf2d0 0x7fc7f40cd490 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 -- 172.21.15.148:0/2170806064 >> 172.21.15.148:0/2170806064 conn(0x7fc7f40cc970 msgr2=0x7fc7f40ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 -- 172.21.15.148:0/2170806064 shutdown_connections 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.797+0000 7fc7f9236640 1 -- 172.21.15.148:0/2170806064 wait complete. 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.799+0000 7fc7f9236640 1 Processor -- start 2024-03-04T12:12:25.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.799+0000 7fc7f9236640 1 -- start start 2024-03-04T12:12:25.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.799+0000 7fc7f9236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7f40af420 0x7fc7f40c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.799+0000 7fc7f9236640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 0x7fc7f40c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.799+0000 7fc7f9236640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f9236640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f406ae70 con 0x7fc7f40c9d00 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f9236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40ceee0 con 0x7fc7f40af420 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f9236640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f409e2d0 con 0x7fc7f40cf2d0 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45450/0 (socket says 172.21.15.148:45450) 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 -- 172.21.15.148:0/3936718517 learned_addr learned my addr 172.21.15.148:0/3936718517 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f2d9d640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7f40af420 0x7fc7f40c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f259c640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 0x7fc7f40c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 -- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 msgr2=0x7fc7f40c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 0x7fc7f40c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 -- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7f40af420 msgr2=0x7fc7f40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7f40af420 0x7fc7f40c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 -- 172.21.15.148:0/3936718517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40a1210 con 0x7fc7f40c9d00 2024-03-04T12:12:25.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f259c640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 0x7fc7f40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f359e640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c7a0 tx=0x7fc7ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec003b70 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec003d30 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f40c4650 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00aa30 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.800+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f40a0ea0 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.801+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f40c4b70 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.802+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc7ec00abd0 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.802+0000 7fc7eb7fe640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 0x7fc7d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.802+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7ec08a3a0 con 0x7fc7f40c9d00 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.802+0000 7fc7f2d9d640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 0x7fc7d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:25.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.803+0000 7fc7f2d9d640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 0x7fc7d406d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4036000 tx=0x7fc7e403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:25.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.807+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7ec058ee0 con 0x7fc7f40c9d00 2024-03-04T12:12:25.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.979+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7f4074f60 con 0x7fc7f40c9d00 2024-03-04T12:12:25.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.979+0000 7fc7eb7fe640 1 -- 172.21.15.148:0/3936718517 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7ec0587e0 con 0x7fc7f40c9d00 2024-03-04T12:12:25.981 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 msgr2=0x7fc7d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 0x7fc7d406d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4036000 tx=0x7fc7e403c040 comp rx=0 tx=0).stop 2024-03-04T12:12:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 msgr2=0x7fc7f41f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c7a0 tx=0x7fc7ec00cc60 comp rx=0 tx=0).stop 2024-03-04T12:12:25.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 shutdown_connections 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc7d406b500 0x7fc7d406d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7f40c9d00 0x7fc7f41f29e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7f40cf2d0 0x7fc7f40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 --2- 172.21.15.148:0/3936718517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7f40af420 0x7fc7f40c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.981+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 >> 172.21.15.148:0/3936718517 conn(0x7fc7f40cc970 msgr2=0x7fc7f40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.982+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 shutdown_connections 2024-03-04T12:12:25.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:25.982+0000 7fc7f9236640 1 -- 172.21.15.148:0/3936718517 wait complete. 2024-03-04T12:12:25.994 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:26.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.096+0000 7fd8b1073640 1 Processor -- start 2024-03-04T12:12:26.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.096+0000 7fd8b1073640 1 -- start start 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8b1073640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8ac161df0 0x7fd8ac0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8b1073640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac162450 con 0x7fd8ac15da20 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8b1073640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac0cc2d0 con 0x7fd8ac0cf660 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8b1073640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac0d0000 con 0x7fd8ac161df0 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8aad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd8ac15da20 0x7fd8ac0ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42560/0 (socket says 172.21.15.148:42560) 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8aad9d640 1 -- 172.21.15.148:0/3060429683 learned_addr learned my addr 172.21.15.148:0/3060429683 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8ab59e640 1 --2- 172.21.15.148:0/3060429683 >> v2:172.21.15.191:3300/0 conn(0x7fd8ac161df0 0x7fd8ac0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586698078 0 0) 0x7fd8ac162450 con 0x7fd8ac15da20 2024-03-04T12:12:26.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.097+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd894003520 con 0x7fd8ac15da20 2024-03-04T12:12:26.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.098+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285023047 0 0) 0x7fd894003520 con 0x7fd8ac15da20 2024-03-04T12:12:26.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.098+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3060429683 >> v1:172.21.15.191:6789/0 conn(0x7fd8ac0cf660 legacy=0x7fd8ac1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.098+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3060429683 >> v1:172.21.15.148:6789/0 conn(0x7fd8ac15da20 legacy=0x7fd8ac0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.098+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3060429683 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0cc5b0 con 0x7fd8ac161df0 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.098+0000 7fd8ab59e640 1 --2- 172.21.15.148:0/3060429683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0cfac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097a0 tx=0x7fd8a402b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c51d63470461d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4037070 con 0x7fd8ac161df0 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4030510 con 0x7fd8ac161df0 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8a9d9b640 1 -- 172.21.15.148:0/3060429683 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0019f0 con 0x7fd8ac161df0 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 -- 172.21.15.148:0/3060429683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac161df0 msgr2=0x7fd8ac0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3060429683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0cfac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097a0 tx=0x7fd8a402b600 comp rx=0 tx=0).stop 2024-03-04T12:12:26.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 -- 172.21.15.148:0/3060429683 shutdown_connections 2024-03-04T12:12:26.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3060429683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0cfac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 -- 172.21.15.148:0/3060429683 >> 172.21.15.148:0/3060429683 conn(0x7fd8ac0b33a0 msgr2=0x7fd8ac0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 -- 172.21.15.148:0/3060429683 shutdown_connections 2024-03-04T12:12:26.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.099+0000 7fd8b1073640 1 -- 172.21.15.148:0/3060429683 wait complete. 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 Processor -- start 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 -- start start 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 0x7fd8ac159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 0x7fd8ac15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15de00 con 0x7fd8ac161df0 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0cc920 con 0x7fd8ac0cf660 2024-03-04T12:12:26.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8b1073640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac06bde0 con 0x7fd8ac15da20 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8ab59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8ab59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45464/0 (socket says 172.21.15.148:45464) 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.101+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3556312625 learned_addr learned my addr 172.21.15.148:0/3556312625 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8aad9d640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 0x7fd8ac159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8aa59c640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 0x7fd8ac15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 msgr2=0x7fd8ac15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 0x7fd8ac15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 msgr2=0x7fd8ac159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 0x7fd8ac159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 -- 172.21.15.148:0/3556312625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac15ede0 con 0x7fd8ac161df0 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8aa59c640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 0x7fd8ac15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8aad9d640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 0x7fd8ac159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:26.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8ab59e640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4035040 tx=0x7fd8a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4042070 con 0x7fd8ac161df0 2024-03-04T12:12:26.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a403e660 con 0x7fd8ac161df0 2024-03-04T12:12:26.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac15f0c0 con 0x7fd8ac161df0 2024-03-04T12:12:26.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4037040 con 0x7fd8ac161df0 2024-03-04T12:12:26.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.102+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac15f680 con 0x7fd8ac161df0 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.103+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac074f60 con 0x7fd8ac161df0 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.103+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd8a4045070 con 0x7fd8ac161df0 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.104+0000 7fd8937fe640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 0x7fd89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.104+0000 7fd8aad9d640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 0x7fd89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.104+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8a40ac360 con 0x7fd8ac161df0 2024-03-04T12:12:26.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.104+0000 7fd8aad9d640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 0x7fd89806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000f80 tx=0x7fd89c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.108+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8a407af40 con 0x7fd8ac161df0 2024-03-04T12:12:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.279+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8ac0b23c0 con 0x7fd8ac161df0 2024-03-04T12:12:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.280+0000 7fd8937fe640 1 -- 172.21.15.148:0/3556312625 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8a407a840 con 0x7fd8ac161df0 2024-03-04T12:12:26.281 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:26.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 msgr2=0x7fd89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 0x7fd89806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000f80 tx=0x7fd89c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 msgr2=0x7fd8ac0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4035040 tx=0x7fd8a40046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 shutdown_connections 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd89806b4d0 0x7fd89806d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac161df0 0x7fd8ac0b5430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8ac15da20 0x7fd8ac15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 --2- 172.21.15.148:0/3556312625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac0cf660 0x7fd8ac159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 >> 172.21.15.148:0/3556312625 conn(0x7fd8ac0b33a0 msgr2=0x7fd8ac0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 shutdown_connections 2024-03-04T12:12:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.282+0000 7fd8b1073640 1 -- 172.21.15.148:0/3556312625 wait complete. 2024-03-04T12:12:26.294 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.393+0000 7fceece36640 1 Processor -- start 2024-03-04T12:12:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.393+0000 7fceece36640 1 -- start start 2024-03-04T12:12:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fceece36640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcee815db80 0x7fcee8071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fceece36640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee80cfb00 0x7fcee806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fceece36640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee80ad560 con 0x7fcee80acda0 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fceece36640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80ad840 con 0x7fcee815db80 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fceece36640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80adb20 con 0x7fcee80cfb00 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcee815db80 0x7fcee8071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcee815db80 0x7fcee8071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45480/0 (socket says 172.21.15.148:45480) 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 -- 172.21.15.148:0/3706514696 learned_addr learned my addr 172.21.15.148:0/3706514696 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee6d9d640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee80cfb00 0x7fcee806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 -- 172.21.15.148:0/3706514696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee80cfb00 msgr2=0x7fcee806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee80cfb00 0x7fcee806d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 -- 172.21.15.148:0/3706514696 >> v1:172.21.15.191:6789/0 conn(0x7fcee80acda0 legacy=0x7fcee80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.394+0000 7fcee659c640 1 -- 172.21.15.148:0/3706514696 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80ade00 con 0x7fcee815db80 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.395+0000 7fcee659c640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee815db80 0x7fcee8071e80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcee8071300 tx=0x7fced802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4582dd4811fbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.395+0000 7fcee5d9b640 1 -- 172.21.15.148:0/3706514696 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8035020 con 0x7fcee815db80 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.395+0000 7fcee5d9b640 1 -- 172.21.15.148:0/3706514696 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced8031830 con 0x7fcee815db80 2024-03-04T12:12:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.395+0000 7fcee5d9b640 1 -- 172.21.15.148:0/3706514696 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8031c00 con 0x7fcee815db80 2024-03-04T12:12:26.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 -- 172.21.15.148:0/3706514696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee815db80 msgr2=0x7fcee8071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee815db80 0x7fcee8071e80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcee8071300 tx=0x7fced802b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 -- 172.21.15.148:0/3706514696 shutdown_connections 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee80cfb00 0x7fcee806d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 --2- 172.21.15.148:0/3706514696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee815db80 0x7fcee8071e80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 -- 172.21.15.148:0/3706514696 >> 172.21.15.148:0/3706514696 conn(0x7fcee806acd0 msgr2=0x7fcee806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 -- 172.21.15.148:0/3706514696 shutdown_connections 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.396+0000 7fceece36640 1 -- 172.21.15.148:0/3706514696 wait complete. 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.397+0000 7fceece36640 1 Processor -- start 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.397+0000 7fceece36640 1 -- start start 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee80cfb00 0x7fcee81f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee815db80 0x7fcee81f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee815df60 con 0x7fcee80acda0 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80ad200 con 0x7fcee80cfb00 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee806dc10 con 0x7fcee815db80 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45496/0 (socket says 172.21.15.148:45496) 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 -- 172.21.15.148:0/3822622025 learned_addr learned my addr 172.21.15.148:0/3822622025 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcedffff640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee80cfb00 0x7fcee81f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 -- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee815db80 msgr2=0x7fcee81f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee815db80 0x7fcee81f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 -- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee80cfb00 msgr2=0x7fcee81f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee80cfb00 0x7fcee81f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 -- 172.21.15.148:0/3822622025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee81f4980 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcee659c640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fced802b910 tx=0x7fced80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced803aa30 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced803abd0 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee815e1f0 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80443f0 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.398+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee815e7b0 con 0x7fcee80acda0 2024-03-04T12:12:26.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.399+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee8074f60 con 0x7fcee80acda0 2024-03-04T12:12:26.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.400+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fced8038020 con 0x7fcee80acda0 2024-03-04T12:12:26.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.400+0000 7fcedeffd640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 0x7fcec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.400+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fced80ac2c0 con 0x7fcee80acda0 2024-03-04T12:12:26.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.400+0000 7fcedffff640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 0x7fcec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.401+0000 7fcedffff640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 0x7fcec806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcee8159920 tx=0x7fced0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.405+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fced803f070 con 0x7fcee80acda0 2024-03-04T12:12:26.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.608+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcee8068030 con 0x7fcee80acda0 2024-03-04T12:12:26.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.608+0000 7fcedeffd640 1 -- 172.21.15.148:0/3822622025 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fced807ae00 con 0x7fcee80acda0 2024-03-04T12:12:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:26.610 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 msgr2=0x7fcec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 0x7fcec806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcee8159920 tx=0x7fced0009040 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 msgr2=0x7fcee81f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fced802b910 tx=0x7fced80046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 shutdown_connections 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcec806b4d0 0x7fcec806d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee815db80 0x7fcee81f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee80cfb00 0x7fcee81f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 --2- 172.21.15.148:0/3822622025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcee80acda0 0x7fcee81f4440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 >> 172.21.15.148:0/3822622025 conn(0x7fcee806acd0 msgr2=0x7fcee80b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 shutdown_connections 2024-03-04T12:12:26.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.610+0000 7fceece36640 1 -- 172.21.15.148:0/3822622025 wait complete. 2024-03-04T12:12:26.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:26.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.723+0000 7f87916fe640 1 Processor -- start 2024-03-04T12:12:26.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.723+0000 7f87916fe640 1 -- start start 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f87916fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f878c15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f87916fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c1624e0 con 0x7f878c0cf650 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f87916fe640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c0cd010 con 0x7f878c0cfab0 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f87916fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c071df0 con 0x7f878c154a80 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f8783fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f878c0cf650 0x7f878c1616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42564/0 (socket says 172.21.15.148:42564) 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f8783fff640 1 -- 172.21.15.148:0/1903751111 learned_addr learned my addr 172.21.15.148:0/1903751111 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878affd640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f878c15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639373946 0 0) 0x7f878c1624e0 con 0x7f878c0cf650 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8778003520 con 0x7f878c0cf650 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362638626 0 0) 0x7f8778003520 con 0x7f878c0cf650 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 msgr2=0x7f878c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f878c15e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 >> v1:172.21.15.191:6789/0 conn(0x7f878c0cfab0 legacy=0x7f878c161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0cdaa0 con 0x7f878c0cf650 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f8774001670 con 0x7f878c0cf650 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 >> v1:172.21.15.148:6789/0 conn(0x7f878c0cf650 legacy=0x7f878c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f87780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778003f20 0x7f8778004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8778004d40 0x7f87780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c071df0 con 0x7f8778003f20 2024-03-04T12:12:26.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877800ba20 con 0x7f8778004d40 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877800bc20 con 0x7f878c154a80 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f8783fff640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778003f20 0x7f8778004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.724+0000 7f878affd640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f87780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f878b7fe640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8778004d40 0x7f87780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 -- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 msgr2=0x7f87780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878c154a80 0x7f87780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 -- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8778004d40 msgr2=0x7f87780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8778004d40 0x7f87780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 -- 172.21.15.148:0/1903751111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778003520 con 0x7f8778003f20 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f878b7fe640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8778004d40 0x7f87780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.725+0000 7f8783fff640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778003f20 0x7f8778004800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8774007fd0 tx=0x7f8778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff7156aed62b6d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774007ca0 con 0x7f8778003f20 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8774030070 con 0x7f8778003f20 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f878a7fc640 1 -- 172.21.15.148:0/1903751111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774005d10 con 0x7f8778003f20 2024-03-04T12:12:26.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 -- 172.21.15.148:0/1903751111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778003f20 msgr2=0x7f8778004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 --2- 172.21.15.148:0/1903751111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778003f20 0x7f8778004800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8774007fd0 tx=0x7f8778005830 comp rx=0 tx=0).stop 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f878b7fe640 1 -- 172.21.15.148:0/1903751111 reap_dead start 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 -- 172.21.15.148:0/1903751111 shutdown_connections 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 -- 172.21.15.148:0/1903751111 >> 172.21.15.148:0/1903751111 conn(0x7f878c06f5f0 msgr2=0x7f878c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 -- 172.21.15.148:0/1903751111 shutdown_connections 2024-03-04T12:12:26.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.726+0000 7f87916fe640 1 -- 172.21.15.148:0/1903751111 wait complete. 2024-03-04T12:12:26.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.727+0000 7f87916fe640 1 Processor -- start 2024-03-04T12:12:26.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.727+0000 7f87916fe640 1 -- start start 2024-03-04T12:12:26.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 0x7f878c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 0x7f878c0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0ce340 con 0x7f878c0c7320 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0ce060 con 0x7f878c0c5360 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f87916fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0ce620 con 0x7f8778003f20 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45504/0 (socket says 172.21.15.148:45504) 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 -- 172.21.15.148:0/107920178 learned_addr learned my addr 172.21.15.148:0/107920178 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f878affd640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 0x7f878c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f878b7fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 0x7f878c0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 -- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 msgr2=0x7f878c1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 0x7f878c1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 -- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 msgr2=0x7f878c0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 0x7f878c0c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f8783fff640 1 -- 172.21.15.148:0/107920178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0c7b00 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f878affd640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 0x7f878c1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.728+0000 7f878b7fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 0x7f878c0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f8783fff640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f877400c160 tx=0x7f878c0cdc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774045070 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8774007c80 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c0c7de0 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877402cbc0 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.729+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c0c92b0 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.730+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c0c8300 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.730+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8774007070 con 0x7f878c0c7320 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.731+0000 7f87837fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 0x7f876c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.731+0000 7f878affd640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 0x7f876c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:26.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.731+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f87740ab240 con 0x7f878c0c7320 2024-03-04T12:12:26.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.731+0000 7f878affd640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 0x7f876c06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f878c0c8510 tx=0x7f877c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:26.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.735+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f878c0c8300 con 0x7f878c0c7320 2024-03-04T12:12:26.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.910+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f878c0b22a0 con 0x7f878c0c7320 2024-03-04T12:12:26.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.911+0000 7f87837fe640 1 -- 172.21.15.148:0/107920178 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18574 (secure 0 0 0) 0x7f8774079d80 con 0x7f878c0c7320 2024-03-04T12:12:26.913 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:11:59.272429+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 msgr2=0x7f876c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 0x7f876c06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f878c0c8510 tx=0x7f877c008040 comp rx=0 tx=0).stop 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 msgr2=0x7f878c1530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f877400c160 tx=0x7f878c0cdc70 comp rx=0 tx=0).stop 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 shutdown_connections 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f876c06b4d0 0x7f876c06d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f878c0c5360 0x7f878c0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c0c7320 0x7f878c1530b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 --2- 172.21.15.148:0/107920178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8778003f20 0x7f878c1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:26.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 >> 172.21.15.148:0/107920178 conn(0x7f878c06f5f0 msgr2=0x7f878c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:26.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 shutdown_connections 2024-03-04T12:12:26.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:26.913+0000 7f87916fe640 1 -- 172.21.15.148:0/107920178 wait complete. 2024-03-04T12:12:26.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.027+0000 7f29c684b640 1 Processor -- start 2024-03-04T12:12:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.027+0000 7f29c684b640 1 -- start start 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.027+0000 7f29c684b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29c015de00 0x7f29c00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c684b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 0x7f29c00b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c684b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b70d0 0x7f29c00b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c684b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00d2820 con 0x7f29c015de00 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c684b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c015e1e0 con 0x7f29c00b9380 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c684b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00d1c10 con 0x7f29c00b70d0 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29c015de00 0x7f29c00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29c015de00 0x7f29c00b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45516/0 (socket says 172.21.15.148:45516) 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 -- 172.21.15.148:0/2569061343 learned_addr learned my addr 172.21.15.148:0/2569061343 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bf7fe640 1 --2- 172.21.15.148:0/2569061343 >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 0x7f29c00b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29c4de8640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b70d0 0x7f29c00b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 -- 172.21.15.148:0/2569061343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b70d0 msgr2=0x7f29c00b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b70d0 0x7f29c00b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 -- 172.21.15.148:0/2569061343 >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 msgr2=0x7f29c00b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 --2- 172.21.15.148:0/2569061343 >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 0x7f29c00b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bffff640 1 -- 172.21.15.148:0/2569061343 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00d1ef0 con 0x7f29c015de00 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.028+0000 7f29bf7fe640 1 --2- 172.21.15.148:0/2569061343 >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 0x7f29c00b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29bffff640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c015de00 0x7f29c00b2e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f29c0161830 tx=0x7f29b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338f2d492b902a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29beffd640 1 -- 172.21.15.148:0/2569061343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035020 con 0x7f29c015de00 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29beffd640 1 -- 172.21.15.148:0/2569061343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4031870 con 0x7f29c015de00 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29beffd640 1 -- 172.21.15.148:0/2569061343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4031c40 con 0x7f29c015de00 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 -- 172.21.15.148:0/2569061343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c015de00 msgr2=0x7f29c00b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c015de00 0x7f29c00b2e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f29c0161830 tx=0x7f29b402b910 comp rx=0 tx=0).stop 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 -- 172.21.15.148:0/2569061343 shutdown_connections 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b70d0 0x7f29c00b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 --2- 172.21.15.148:0/2569061343 >> v2:172.21.15.191:3300/0 conn(0x7f29c00b9380 0x7f29c00b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 --2- 172.21.15.148:0/2569061343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c015de00 0x7f29c00b2e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.029+0000 7f29c684b640 1 -- 172.21.15.148:0/2569061343 >> 172.21.15.148:0/2569061343 conn(0x7f29c00d0470 msgr2=0x7f29c00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.030+0000 7f29c684b640 1 -- 172.21.15.148:0/2569061343 shutdown_connections 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.030+0000 7f29c684b640 1 -- 172.21.15.148:0/2569061343 wait complete. 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 Processor -- start 2024-03-04T12:12:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 -- start start 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b9380 0x7f29c0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29c015de00 0x7f29c015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00b97e0 con 0x7f29c00b70d0 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00d24b0 con 0x7f29c015de00 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.031+0000 7f29c684b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00b59b0 con 0x7f29c00b9380 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45518/0 (socket says 172.21.15.148:45518) 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 -- 172.21.15.148:0/510768809 learned_addr learned my addr 172.21.15.148:0/510768809 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 -- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b9380 msgr2=0x7f29c0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b9380 0x7f29c0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 -- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29c015de00 msgr2=0x7f29c015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29c4de8640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29c015de00 0x7f29c015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29c015de00 0x7f29c015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 -- 172.21.15.148:0/510768809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0155260 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bffff640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f29b402b950 tx=0x7f29b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4040070 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4030d00 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c0155540 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035040 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.032+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c0155b00 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.033+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0074f60 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.034+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f29b404b060 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.034+0000 7f29bcff9640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 0x7f29ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.034+0000 7f29bf7fe640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 0x7f29ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.034+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f29b40ac280 con 0x7f29c00b70d0 2024-03-04T12:12:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.035+0000 7f29bf7fe640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 0x7f29ac06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.039+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29b407adc0 con 0x7f29c00b70d0 2024-03-04T12:12:27.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.244+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c00b7a80 con 0x7f29c00b70d0 2024-03-04T12:12:27.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.244+0000 7f29bcff9640 1 -- 172.21.15.148:0/510768809 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f29b407a6c0 con 0x7f29c00b70d0 2024-03-04T12:12:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:27.246 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:27.246 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 msgr2=0x7f29ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 0x7f29ac06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0008040 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 msgr2=0x7f29c014ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f29b402b950 tx=0x7f29b40046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 shutdown_connections 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f29ac06b500 0x7f29ac06d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29c015de00 0x7f29c015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29c00b9380 0x7f29c0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 --2- 172.21.15.148:0/510768809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29c00b70d0 0x7f29c014ddc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 >> 172.21.15.148:0/510768809 conn(0x7f29c00d0470 msgr2=0x7f29c00ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 shutdown_connections 2024-03-04T12:12:27.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.246+0000 7f29c684b640 1 -- 172.21.15.148:0/510768809 wait complete. 2024-03-04T12:12:27.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T12:11:54.225836+0000', 'modified': '2024-03-04T12:12:16.303042+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/2190957164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6833', 'nonce': 2190957164}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 2190957164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-04T12:12:27.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T12:12:27.661 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T12:12:27.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.765+0000 7fcf992c0640 1 Processor -- start 2024-03-04T12:12:27.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.765+0000 7fcf992c0640 1 -- start start 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf992c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf940b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf992c0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9415e1e0 con 0x7fcf940b9380 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf992c0640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf940d1c10 con 0x7fcf940b5470 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf992c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d2820 con 0x7fcf9415de00 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf927fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcf940b9380 0x7fcf940b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42578/0 (socket says 172.21.15.148:42578) 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf927fc640 1 -- 172.21.15.148:0/1657162734 learned_addr learned my addr 172.21.15.148:0/1657162734 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351787032 0 0) 0x7fcf9415e1e0 con 0x7fcf940b9380 2024-03-04T12:12:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf940b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf7c003520 con 0x7fcf940b9380 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063175847 0 0) 0x7fcf7c003520 con 0x7fcf940b9380 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 msgr2=0x7fcf940b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf940b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 >> v1:172.21.15.191:6789/0 conn(0x7fcf940b5470 legacy=0x7fcf940d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940d1ef0 con 0x7fcf940b9380 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.766+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fcf88001670 con 0x7fcf940b9380 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 >> v1:172.21.15.148:6789/0 conn(0x7fcf940b9380 legacy=0x7fcf940b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf7c0039e0 0x7fcf7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d2820 con 0x7fcf7c004d40 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00ba70 con 0x7fcf7c0039e0 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00bc70 con 0x7fcf9415de00 2024-03-04T12:12:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf927fc640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf7c0039e0 0x7fcf7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 -- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 msgr2=0x7fcf7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf9415de00 0x7fcf7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 -- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf7c0039e0 msgr2=0x7fcf7c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf7c0039e0 0x7fcf7c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.767+0000 7fcf937fe640 1 -- 172.21.15.148:0/1657162734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c003520 con 0x7fcf7c004d40 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf937fe640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001720 tx=0x7fcf7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901f40f327477e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c001370 con 0x7fcf7c004d40 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0052d0 con 0x7fcf7c004d40 2024-03-04T12:12:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf91ffb640 1 -- 172.21.15.148:0/1657162734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c003830 con 0x7fcf7c004d40 2024-03-04T12:12:27.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 -- 172.21.15.148:0/1657162734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c004d40 msgr2=0x7fcf7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 --2- 172.21.15.148:0/1657162734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001720 tx=0x7fcf7c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:27.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf937fe640 1 -- 172.21.15.148:0/1657162734 reap_dead start 2024-03-04T12:12:27.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 -- 172.21.15.148:0/1657162734 shutdown_connections 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 -- 172.21.15.148:0/1657162734 >> 172.21.15.148:0/1657162734 conn(0x7fcf940d0470 msgr2=0x7fcf940d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 -- 172.21.15.148:0/1657162734 shutdown_connections 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.768+0000 7fcf992c0640 1 -- 172.21.15.148:0/1657162734 wait complete. 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.770+0000 7fcf992c0640 1 Processor -- start 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.770+0000 7fcf992c0640 1 -- start start 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.770+0000 7fcf992c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.770+0000 7fcf992c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf940cc050 0x7fcf940aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf940aa910 0x7fcf9414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940b97e0 con 0x7fcf940b1e70 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d24b0 con 0x7fcf940cc050 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940b72a0 con 0x7fcf940aa910 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45534/0 (socket says 172.21.15.148:45534) 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 -- 172.21.15.148:0/519720254 learned_addr learned my addr 172.21.15.148:0/519720254 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf937fe640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf940aa910 0x7fcf9414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf927fc640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf940cc050 0x7fcf940aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 -- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf940aa910 msgr2=0x7fcf9414cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf940aa910 0x7fcf9414cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 -- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf940cc050 msgr2=0x7fcf940aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf940cc050 0x7fcf940aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 -- 172.21.15.148:0/519720254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940b2250 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf84011e30 tx=0x7fcf8400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf7b7fe640 1 -- 172.21.15.148:0/519720254 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84015630 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf7b7fe640 1 -- 172.21.15.148:0/519720254 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf840157d0 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf940cc4d0 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf7b7fe640 1 -- 172.21.15.148:0/519720254 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84015ba0 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.771+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf940cca90 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.773+0000 7fcf7b7fe640 1 -- 172.21.15.148:0/519720254 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcf8401d400 con 0x7fcf940b1e70 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.773+0000 7fcf7b7fe640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 0x7fcf8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.773+0000 7fcf7b7fe640 1 -- 172.21.15.148:0/519720254 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcf8408b740 con 0x7fcf940b1e70 2024-03-04T12:12:27.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.773+0000 7fcf927fc640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 0x7fcf8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.774+0000 7fcf927fc640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 0x7fcf8006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf940b90d0 tx=0x7fcf88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.774+0000 7fcf9bf7c740 1 Processor -- start 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.774+0000 7fcf9bf7c740 1 -- start start 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12bf280 0x5626a12c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a1292380 0x5626a12c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a11fc7c0 con 0x5626a12beea0 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a11a5b50 con 0x5626a1292380 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a11f8860 con 0x5626a12bf280 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45540/0 (socket says 172.21.15.148:45540) 2024-03-04T12:12:27.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 -- 172.21.15.148:0/3594973766 learned_addr learned my addr 172.21.15.148:0/3594973766 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 -- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12bf280 msgr2=0x5626a12c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf927fc640 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12bf280 0x5626a12c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12bf280 0x5626a12c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 -- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a1292380 msgr2=0x5626a12c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a1292380 0x5626a12c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.775+0000 7fcf92ffd640 1 -- 172.21.15.148:0/3594973766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a11b99a0 con 0x5626a12beea0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf84010010 tx=0x7fcf8405dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7616e8dfdbffc0d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf78ff9640 1 -- 172.21.15.148:0/3594973766 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84057fc0 con 0x5626a12beea0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf78ff9640 1 -- 172.21.15.148:0/3594973766 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84056330 con 0x5626a12beea0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf78ff9640 1 -- 172.21.15.148:0/3594973766 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8406ddf0 con 0x5626a12beea0 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/3594973766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 msgr2=0x5626a1291e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf84010010 tx=0x7fcf8405dd60 comp rx=0 tx=0).stop 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/3594973766 shutdown_connections 2024-03-04T12:12:27.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a1292380 0x5626a12c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12bf280 0x5626a12c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/3594973766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a1291e40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.776+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/3594973766 >> 172.21.15.148:0/3594973766 conn(0x5626a12940b0 msgr2=0x5626a12dcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.777+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/3594973766 shutdown_connections 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.777+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/3594973766 wait complete. 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.777+0000 7fcf9bf7c740 1 -- start start 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.777+0000 7fcf9bf7c740 1 Processor -- start 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a1296750 con 0x5626a12bf280 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a11b8f90 con 0x5626a12beea0 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a1298f90 con 0x5626a12dada0 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf9bf7c740 10 client.0 _ll_register_callbacks cb 0x5626a11dad90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:12:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45550/0 (socket says 172.21.15.148:45550) 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 -- 172.21.15.148:0/2593065149 learned_addr learned my addr 172.21.15.148:0/2593065149 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf937fe640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 -- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 msgr2=0x5626a12db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 -- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 msgr2=0x5626a12ecdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12ecdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 -- 172.21.15.148:0/2593065149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626a0f112d0 con 0x5626a12bf280 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf88015780 tx=0x7fcf940d2790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f29e9117462bbc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.778+0000 7fcf71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf880148d0 con 0x5626a12bf280 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcf88014a70 con 0x5626a12bf280 2024-03-04T12:12:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf84056330 con 0x5626a12bf280 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf8802e050 con 0x5626a12bf280 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 10 client.4863 fetch_fsmap learned FSMap version 8 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5626a0fdb6b0 con 0x5626a12bf280 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84056330 con 0x5626a12bf280 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fcf88031020 con 0x5626a12bf280 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 10 client.4863 fetch_fsmap finished waiting for FSMap version 8 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 10 client.4863 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 20 client.4863 populate_metadata read hostname 'smithi148' 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a12fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 4 client.4863 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 --> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626a1209d00 con 0x5626a12f92d0 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.779+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a12fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf71ffb640 10 client.4863 ms_handle_connect on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a12fb6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2681aff97ebf4ab server_cookie=851795c28a7625c9 in_seq=0 out_seq=0 2024-03-04T12:12:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2593065149 <== mds.0 v2:172.21.15.191:6833/2190957164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626a1209d00 con 0x5626a12f92d0 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf71ffb640 10 client.4863 handle_command_reply: tid=0 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 client.4863 shutdown 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 msgr2=0x5626a12c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf88015780 tx=0x7fcf940d2790 comp rx=0 tx=0).stop 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 shutdown_connections 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a12fb6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a12c6560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf71ffb640 0 client.4863 ms_handle_reset on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2593065149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12ecdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.780+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 >> 172.21.15.148:0/2593065149 conn(0x5626a1292380 msgr2=0x5626a1295d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.781+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 shutdown_connections 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.781+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2593065149 wait complete. 2024-03-04T12:12:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.781+0000 7fcf9bf7c740 20 client.4863 trim_cache size 0 max 16384 2024-03-04T12:12:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.796+0000 7fcf9bf7c740 1 Processor -- start 2024-03-04T12:12:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.796+0000 7fcf9bf7c740 1 -- start start 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 0x5626a12fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf942012d0 con 0x5626a12beea0 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94200700 con 0x5626a12bf280 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94200d00 con 0x5626a12dada0 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45566/0 (socket says 172.21.15.148:45566) 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 -- 172.21.15.148:0/2921513111 learned_addr learned my addr 172.21.15.148:0/2921513111 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf937fe640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 0x5626a12fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 -- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 msgr2=0x5626a12e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 -- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 msgr2=0x5626a12fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 0x5626a12fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 -- 172.21.15.148:0/2921513111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a12c68d0 con 0x5626a12beea0 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf937fe640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 0x5626a12fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf84059c50 tx=0x7fcf8408a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5683faee0b8fb223 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf73fff640 1 -- 172.21.15.148:0/2921513111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84057fc0 con 0x5626a12beea0 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf73fff640 1 -- 172.21.15.148:0/2921513111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8406dbd0 con 0x5626a12beea0 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.797+0000 7fcf73fff640 1 -- 172.21.15.148:0/2921513111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84061e90 con 0x5626a12beea0 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2921513111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 msgr2=0x5626a12dbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf84059c50 tx=0x7fcf8408a000 comp rx=0 tx=0).stop 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2921513111 shutdown_connections 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12bf280 0x5626a12fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2921513111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12beea0 0x5626a12dbc40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2921513111 >> 172.21.15.148:0/2921513111 conn(0x5626a1292380 msgr2=0x5626a12c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2921513111 shutdown_connections 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2921513111 wait complete. 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.798+0000 7fcf9bf7c740 1 -- start start 2024-03-04T12:12:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 Processor -- start 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a12b1820 con 0x5626a12bf280 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a12b13d0 con 0x5626a12beea0 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626a12b1e20 con 0x5626a12dada0 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf9bf7c740 10 client.0 _ll_register_callbacks cb 0x7fcf84009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45574/0 (socket says 172.21.15.148:45574) 2024-03-04T12:12:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf927fc640 1 -- 172.21.15.148:0/2673470291 learned_addr learned my addr 172.21.15.148:0/2673470291 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.799+0000 7fcf937fe640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 -- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 msgr2=0x5626a12c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 -- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 msgr2=0x5626a12eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 -- 172.21.15.148:0/2673470291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626a12dc720 con 0x5626a12bf280 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf927fc640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf88004840 tx=0x7fcf88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3683f5d5c8f24d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf880297a0 con 0x5626a12bf280 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcf88029940 con 0x5626a12bf280 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf8406dbd0 con 0x5626a12bf280 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf88043050 con 0x5626a12bf280 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 10 client.4869 fetch_fsmap learned FSMap version 8 2024-03-04T12:12:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5626a12f1830 con 0x5626a12bf280 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8406dbd0 con 0x5626a12bf280 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fcf88046020 con 0x5626a12bf280 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 10 client.4869 fetch_fsmap finished waiting for FSMap version 8 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 10 client.4869 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 20 client.4869 populate_metadata read hostname 'smithi148' 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a134b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 4 client.4869 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.800+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 --> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626a1204050 con 0x5626a12f92d0 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a134b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf92ffd640 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a134b680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83bc975d9636edf server_cookie=1eb9a89be2f2f243 in_seq=0 out_seq=0 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf71ffb640 10 client.4869 ms_handle_connect on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf71ffb640 1 -- 172.21.15.148:0/2673470291 <== mds.0 v2:172.21.15.191:6833/2190957164 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5626a1204050 con 0x5626a12f92d0 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf71ffb640 10 client.4869 handle_command_reply: tid=0 2024-03-04T12:12:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 client.4869 shutdown 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 msgr2=0x5626a1345670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf88004840 tx=0x7fcf88016000 comp rx=0 tx=0).stop 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 shutdown_connections 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:6833/2190957164,v1:172.21.15.191:6835/2190957164] conn(0x5626a12f92d0 0x5626a134b680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5626a12dada0 0x5626a12c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf71ffb640 0 client.4869 ms_handle_reset on v2:172.21.15.191:6833/2190957164 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5626a12bf280 0x5626a1345670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 --2- 172.21.15.148:0/2673470291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5626a12beea0 0x5626a12eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 >> 172.21.15.148:0/2673470291 conn(0x5626a1292380 msgr2=0x5626a12e03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 shutdown_connections 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.801+0000 7fcf9bf7c740 1 -- 172.21.15.148:0/2673470291 wait complete. 2024-03-04T12:12:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf9bf7c740 20 client.4869 trim_cache size 0 max 16384 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "d4b3ed31-a94c-42d0-968e-1d6a4e199a28", 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4288, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 32.906022802999999, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 8, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 24, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 24, 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 39.423657505000001 2024-03-04T12:12:27.810 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-04T12:12:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 msgr2=0x7fcf8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 0x7fcf8006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf940b90d0 tx=0x7fcf88009040 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 msgr2=0x7fcf940cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf84011e30 tx=0x7fcf8400f580 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 shutdown_connections 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fcf8006b4d0 0x7fcf8006d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf940aa910 0x7fcf9414cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf940cc050 0x7fcf940aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 --2- 172.21.15.148:0/519720254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf940b1e70 0x7fcf940cbb10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 >> 172.21.15.148:0/519720254 conn(0x7fcf940d0470 msgr2=0x7fcf940cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 shutdown_connections 2024-03-04T12:12:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:27.802+0000 7fcf992c0640 1 -- 172.21.15.148:0/519720254 wait complete. 2024-03-04T12:12:27.814 INFO:teuthology.run_tasks:Running task kclient... 2024-03-04T12:12:27.827 INFO:tasks.kclient:Mounting kernel clients... 2024-03-04T12:12:27.827 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-04T12:12:27.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:27.827 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:12:27.842 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-04T12:12:27.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:27.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T12:12:27.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:27.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-03-04T12:12:28.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.014 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:12:28.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-03-04T12:12:28.099 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-03-04T12:12:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:12:28.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.101 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-04T12:12:28.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.119 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-03-04T12:12:28.192 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2024-03-04T12:12:28.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:12:28.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.193 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-04T12:12:28.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-03-04T12:12:28.320 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2024-03-04T12:12:28.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T12:12:28.322 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-04T12:12:28.322 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T12:12:28.322 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T12:12:28.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:12:28.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 Processor -- start 2024-03-04T12:12:28.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 -- start start 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd38c15b030 0x7fd38c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c06d8b0 con 0x7fd38c06d3b0 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c06db90 con 0x7fd38c15b410 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.423+0000 7fd392b29640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0b2720 con 0x7fd38c15b030 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd3908c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd38c15b030 0x7fd38c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd38bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd38c06d3b0 0x7fd38c15dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42590/0 (socket says 172.21.15.148:42590) 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd38bfff640 1 -- 172.21.15.148:0/4286816257 learned_addr learned my addr 172.21.15.148:0/4286816257 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd3908c5640 1 -- 172.21.15.148:0/4286816257 >> v1:172.21.15.191:6789/0 conn(0x7fd38c15b410 legacy=0x7fd38c15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd3908c5640 1 -- 172.21.15.148:0/4286816257 >> v1:172.21.15.148:6789/0 conn(0x7fd38c06d3b0 legacy=0x7fd38c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.424+0000 7fd3908c5640 1 -- 172.21.15.148:0/4286816257 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c06de70 con 0x7fd38c15b030 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd3908c5640 1 --2- 172.21.15.148:0/4286816257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c15c120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3800097a0 tx=0x7fd38002b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff98369ec8969d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd38b7fe640 1 -- 172.21.15.148:0/4286816257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380037070 con 0x7fd38c15b030 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd38b7fe640 1 -- 172.21.15.148:0/4286816257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380030710 con 0x7fd38c15b030 2024-03-04T12:12:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd38b7fe640 1 -- 172.21.15.148:0/4286816257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380030ae0 con 0x7fd38c15b030 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 -- 172.21.15.148:0/4286816257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 msgr2=0x7fd38c15c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 --2- 172.21.15.148:0/4286816257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c15c120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3800097a0 tx=0x7fd38002b680 comp rx=0 tx=0).stop 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 -- 172.21.15.148:0/4286816257 shutdown_connections 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 --2- 172.21.15.148:0/4286816257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c15c120 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 -- 172.21.15.148:0/4286816257 >> 172.21.15.148:0/4286816257 conn(0x7fd38c06acd0 msgr2=0x7fd38c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 -- 172.21.15.148:0/4286816257 shutdown_connections 2024-03-04T12:12:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.425+0000 7fd392b29640 1 -- 172.21.15.148:0/4286816257 wait complete. 2024-03-04T12:12:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.426+0000 7fd392b29640 1 Processor -- start 2024-03-04T12:12:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 -- start start 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 0x7fd38c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd38c15b410 0x7fd38c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c06e810 con 0x7fd38c15b030 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c06e690 con 0x7fd38c06d3b0 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd392b29640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b2bb0 con 0x7fd38c15b410 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.427+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45588/0 (socket says 172.21.15.148:45588) 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 -- 172.21.15.148:0/2086723194 learned_addr learned my addr 172.21.15.148:0/2086723194 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:28.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3908c5640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 0x7fd38c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3910c6640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd38c15b410 0x7fd38c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 -- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd38c15b410 msgr2=0x7fd38c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd38c15b410 0x7fd38c15d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 -- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 msgr2=0x7fd38c0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 0x7fd38c0cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 -- 172.21.15.148:0/2086723194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c1f2450 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3908c5640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 0x7fd38c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd38bfff640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c780 tx=0x7fd37800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378018070 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780072d0 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c1f2730 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780047e0 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.428+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1483e0 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.429+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c074f60 con 0x7fd38c15b030 2024-03-04T12:12:28.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.430+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd37801b070 con 0x7fd38c15b030 2024-03-04T12:12:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.430+0000 7fd3897fa640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 0x7fd37006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.430+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd378089950 con 0x7fd38c15b030 2024-03-04T12:12:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.430+0000 7fd3908c5640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 0x7fd37006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.431+0000 7fd3908c5640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 0x7fd37006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd380034000 tx=0x7fd380008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:28.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.434+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd378058520 con 0x7fd38c15b030 2024-03-04T12:12:28.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.640+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd38c0aec30 con 0x7fd38c15b030 2024-03-04T12:12:28.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.641+0000 7fd3897fa640 1 -- 172.21.15.148:0/2086723194 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd3780012b0 con 0x7fd38c15b030 2024-03-04T12:12:28.643 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T12:12:28.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 msgr2=0x7fd37006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 0x7fd37006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd380034000 tx=0x7fd380008030 comp rx=0 tx=0).stop 2024-03-04T12:12:28.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 msgr2=0x7fd38c0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c780 tx=0x7fd37800cc40 comp rx=0 tx=0).stop 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 shutdown_connections 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd37006b4d0 0x7fd37006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd38c15b410 0x7fd38c15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c15b030 0x7fd38c0ce390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 --2- 172.21.15.148:0/2086723194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd38c06d3b0 0x7fd38c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 >> 172.21.15.148:0/2086723194 conn(0x7fd38c06acd0 msgr2=0x7fd38c0cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 shutdown_connections 2024-03-04T12:12:28.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:28.643+0000 7fd392b29640 1 -- 172.21.15.148:0/2086723194 wait complete. 2024-03-04T12:12:28.655 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T12:12:28.655 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:12:28.655 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:12:28.656 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T12:12:28.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.656 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42693sec preferred_lft 42693sec 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-04T12:12:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:28.672 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T12:12:28.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T12:12:28.673 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:28.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.780 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:12:28.805 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-04T12:12:28.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.807 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-04T12:12:28.862 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-04T12:12:28.862 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:12:28.862 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:12:28.862 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:12:28.862 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:12:28.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:12:28.863 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:28.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.958 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:12:28.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:28.973 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-04T12:12:29.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.028 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:29.029 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:29.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T12:12:29.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-04T12:12:29.029 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:29.110 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-04T12:12:29.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-04T12:12:29.111 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:29.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.236 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:29.236 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:29.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-04T12:12:29.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T12:12:29.236 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:29.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:29.278 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T12:12:29.333 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T12:12:29.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:29.333 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T12:12:29.387 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-04T12:12:29.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:29.471 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:29.472 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:29.583 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:29.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:29.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-04T12:12:29.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@d4b3ed31-a94c-42d0-968e-1d6a4e199a28.cephfs=/ 2024-03-04T12:12:29.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-04T12:12:29.584 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-04T12:12:29.584 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-03-04T12:12:29.584 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T12:12:29.584 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T12:12:29.584 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T12:12:29.585 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T12:12:29.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.586 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T12:12:29.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.612 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:29.669 INFO:teuthology.orchestra.run.smithi148.stdout:client4559 2024-03-04T12:12:29.669 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:29.669 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:29.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.725 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:29.780 INFO:teuthology.orchestra.run.smithi148.stdout:client4559 2024-03-04T12:12:29.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.781 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:29.837 INFO:teuthology.orchestra.run.smithi148.stdout:client4559 2024-03-04T12:12:29.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:29.838 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d4b3ed31-a94c-42d0-968e-1d6a4e199a28.client4559/status') 2024-03-04T12:12:29.903 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4559 (3)172.21.15.148:0/3419040451 2024-03-04T12:12:29.903 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-03-04T12:12:29.904 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-03-04T12:12:29.904 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-03-04T12:12:29.904 INFO:teuthology.orchestra.run.smithi148.stderr:71 bytes copied, 2.1479e-05 s, 3.3 MB/s 2024-03-04T12:12:29.905 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-04T12:12:29.905 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T12:12:29.905 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-04T12:12:29.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:12:30.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.046+0000 7fef54109640 1 Processor -- start 2024-03-04T12:12:30.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.046+0000 7fef54109640 1 -- start start 2024-03-04T12:12:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef54109640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fef4c0cc500 0x7fef4c166680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef54109640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c0d0140 con 0x7fef4c06d3b0 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef54109640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c15c280 con 0x7fef4c0cfd60 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef54109640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c15cbb0 con 0x7fef4c0cc500 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef516a4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef4c06d3b0 0x7fef4c0ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42596/0 (socket says 172.21.15.148:42596) 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef516a4640 1 -- 172.21.15.148:0/3760320773 learned_addr learned my addr 172.21.15.148:0/3760320773 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef526a6640 1 --2- 172.21.15.148:0/3760320773 >> v2:172.21.15.191:3300/0 conn(0x7fef4c0cc500 0x7fef4c166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774128267 0 0) 0x7fef4c0d0140 con 0x7fef4c06d3b0 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef38003520 con 0x7fef4c06d3b0 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745078474 0 0) 0x7fef4c15c280 con 0x7fef4c0cfd60 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef4c0d0140 con 0x7fef4c0cfd60 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195339324 0 0) 0x7fef38003520 con 0x7fef4c06d3b0 2024-03-04T12:12:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 >> v2:172.21.15.191:3300/0 conn(0x7fef4c0cc500 msgr2=0x7fef4c166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 --2- 172.21.15.148:0/3760320773 >> v2:172.21.15.191:3300/0 conn(0x7fef4c0cc500 0x7fef4c166680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 >> v1:172.21.15.191:6789/0 conn(0x7fef4c0cfd60 legacy=0x7fef4c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c15c560 con 0x7fef4c06d3b0 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fef3c001670 con 0x7fef4c06d3b0 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef526a6640 1 --2- 172.21.15.148:0/3760320773 >> v2:172.21.15.191:3300/0 conn(0x7fef4c0cc500 0x7fef4c166680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 >> v1:172.21.15.148:6789/0 conn(0x7fef4c06d3b0 legacy=0x7fef4c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38003b40 0x7fef38003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 0x7fef38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef380051c0 0x7fef3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c15cbb0 con 0x7fef38004800 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3800c040 con 0x7fef38003b40 2024-03-04T12:12:30.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.047+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3800c240 con 0x7fef380051c0 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 0x7fef38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef51ea5640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38003b40 0x7fef38003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef526a6640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef380051c0 0x7fef3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 -- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef380051c0 msgr2=0x7fef3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef380051c0 0x7fef3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 -- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38003b40 msgr2=0x7fef38003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38003b40 0x7fef38003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef516a4640 1 -- 172.21.15.148:0/3760320773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38003520 con 0x7fef38004800 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef526a6640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef380051c0 0x7fef3800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.048+0000 7fef526a6640 1 -- 172.21.15.148:0/3760320773 reap_dead start 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef516a4640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 0x7fef38004c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007fd0 tx=0x7fef38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769293bfc7fbb75f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:30.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c007ca0 con 0x7fef38004800 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c030070 con 0x7fef38004800 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef50ea3640 1 -- 172.21.15.148:0/3760320773 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c005d50 con 0x7fef38004800 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 -- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 msgr2=0x7fef38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 0x7fef38004c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007fd0 tx=0x7fef38005830 comp rx=0 tx=0).stop 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 -- 172.21.15.148:0/3760320773 shutdown_connections 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 --2- 172.21.15.148:0/3760320773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef38004800 0x7fef38004c80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 -- 172.21.15.148:0/3760320773 >> 172.21.15.148:0/3760320773 conn(0x7fef4c06acd0 msgr2=0x7fef4c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 -- 172.21.15.148:0/3760320773 shutdown_connections 2024-03-04T12:12:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.049+0000 7fef54109640 1 -- 172.21.15.148:0/3760320773 wait complete. 2024-03-04T12:12:30.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 Processor -- start 2024-03-04T12:12:30.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 -- start start 2024-03-04T12:12:30.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 0x7fef4c0a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef4c14a510 0x7fef4c0aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0cbdf0 con 0x7fef4c14a090 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c15bf30 con 0x7fef38004800 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef54109640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0cbf70 con 0x7fef4c14a510 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef516a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef516a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45612/0 (socket says 172.21.15.148:45612) 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.051+0000 7fef516a4640 1 -- 172.21.15.148:0/4091518065 learned_addr learned my addr 172.21.15.148:0/4091518065 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef526a6640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef4c14a510 0x7fef4c0aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef51ea5640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 0x7fef4c0a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 -- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef4c14a510 msgr2=0x7fef4c0aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef4c14a510 0x7fef4c0aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 -- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 msgr2=0x7fef4c0a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 0x7fef4c0a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 -- 172.21.15.148:0/4091518065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c153ab0 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef51ea5640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 0x7fef4c0a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef516a4640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0018f0 tx=0x7fef3c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c045070 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c02cbf0 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c0aa860 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c007ca0 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.052+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0a88c0 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.053+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c074f60 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.054+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fef3c048070 con 0x7fef4c14a090 2024-03-04T12:12:30.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.054+0000 7fef427fc640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:30.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.054+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fef3c0ab2c0 con 0x7fef4c14a090 2024-03-04T12:12:30.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.054+0000 7fef51ea5640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:30.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.054+0000 7fef51ea5640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 0x7fef3006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef4c153cd0 tx=0x7fef48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:30.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.058+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef3c079e90 con 0x7fef4c14a090 2024-03-04T12:12:30.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.266+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef4c15ba60 con 0x7fef4c14a090 2024-03-04T12:12:30.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.266+0000 7fef427fc640 1 -- 172.21.15.148:0/4091518065 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fef3c079790 con 0x7fef4c14a090 2024-03-04T12:12:30.268 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T12:12:30.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 msgr2=0x7fef3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 0x7fef3006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef4c153cd0 tx=0x7fef48009040 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 msgr2=0x7fef4c0a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0018f0 tx=0x7fef3c0019c0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 shutdown_connections 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef4c14a510 0x7fef4c0aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c14a090 0x7fef4c0a9cf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 --2- 172.21.15.148:0/4091518065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef38004800 0x7fef4c0a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 >> 172.21.15.148:0/4091518065 conn(0x7fef4c06acd0 msgr2=0x7fef4c06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 shutdown_connections 2024-03-04T12:12:30.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:30.268+0000 7fef54109640 1 -- 172.21.15.148:0/4091518065 wait complete. 2024-03-04T12:12:30.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T12:12:30.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:12:30.281 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-04T12:12:30.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.281 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-04T12:12:30.297 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42691sec preferred_lft 42691sec 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether da:49:f7:74:bb:9a brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::5c1c:10ff:feb4:9a30/64 scope link tentative 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-04T12:12:30.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether da:49:f7:74:bb:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T12:12:30.299 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::d849:f7ff:fe74:bb9a/64 scope link tentative 2024-03-04T12:12:30.299 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.299 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:12:30.363 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-04T12:12:30.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.365 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-04T12:12:30.420 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-04T12:12:30.420 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:12:30.421 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:12:30.421 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:12:30.421 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:12:30.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:12:30.421 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:30.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.518 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:12:30.532 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-04T12:12:30.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.533 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-04T12:12:30.587 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-04T12:12:30.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.588 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:30.588 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:30.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-04T12:12:30.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-04T12:12:30.588 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:30.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:30.737 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9a:ea:40:6b:e7:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-04T12:12:30.738 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-04T12:12:30.738 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.738 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::98ea:40ff:fe6b:e72c/64 scope link tentative 2024-03-04T12:12:30.738 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.739 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-04T12:12:30.806 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:30.806 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:30.806 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:30.806 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.806 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9a:ea:40:6b:e7:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::98ea:40ff:fe6b:e72c/64 scope link tentative 2024-03-04T12:12:30.807 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:30.808 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-04T12:12:30.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.808 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:30.808 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-04T12:12:30.809 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:30.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:30.935 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:12:30.935 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:12:30.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2024-03-04T12:12:30.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2024-03-04T12:12:30.935 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:12:30.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:30.977 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-04T12:12:31.032 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-04T12:12:31.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:31.032 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-04T12:12:31.086 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-04T12:12:31.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.087 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:31.170 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:31.170 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 2@d4b3ed31-a94c-42d0-968e-1d6a4e199a28.cephfs=/ 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.148:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-04T12:12:31.178 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-04T12:12:31.179 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-03-04T12:12:31.179 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T12:12:31.179 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T12:12:31.179 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T12:12:31.180 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T12:12:31.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-04T12:12:31.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.207 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:31.264 INFO:teuthology.orchestra.run.smithi148.stdout:client4565 2024-03-04T12:12:31.264 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:31.264 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:31.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.319 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:31.376 INFO:teuthology.orchestra.run.smithi148.stdout:client4565 2024-03-04T12:12:31.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.376 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:31.432 INFO:teuthology.orchestra.run.smithi148.stdout:client4565 2024-03-04T12:12:31.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.432 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d4b3ed31-a94c-42d0-968e-1d6a4e199a28.client4565/status') 2024-03-04T12:12:31.499 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4565 (3)172.21.15.148:0/1996283935 2024-03-04T12:12:31.499 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-03-04T12:12:31.500 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-03-04T12:12:31.500 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-03-04T12:12:31.500 INFO:teuthology.orchestra.run.smithi148.stderr:71 bytes copied, 1.9628e-05 s, 3.6 MB/s 2024-03-04T12:12:31.500 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-04T12:12:31.500 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T12:12:31.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-04T12:12:31.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:12:31.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.638+0000 7f4a072ea640 1 Processor -- start 2024-03-04T12:12:31.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 -- start start 2024-03-04T12:12:31.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a0006d3b0 0x7f4a0015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0015be00 con 0x7f4a0015b030 2024-03-04T12:12:31.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000b2720 con 0x7f4a0006d3b0 2024-03-04T12:12:31.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.639+0000 7f4a072ea640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006d8b0 con 0x7f4a0015b410 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a05887640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a0006d3b0 0x7f4a0015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a05086640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4a0015b030 0x7f4a0015dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:47560/0 (socket says 172.21.15.148:47560) 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a0006d3b0 0x7f4a0015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:46576/0 (socket says 172.21.15.148:46576) 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a05887640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:60146/0 (socket says 172.21.15.148:60146) 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a05086640 1 -- 172.21.15.148:0/605088845 learned_addr learned my addr 172.21.15.148:0/605088845 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 -- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 msgr2=0x7f4a0015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 -- 172.21.15.148:0/605088845 >> v1:172.21.15.191:6789/0 conn(0x7f4a0015b030 legacy=0x7f4a0015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a04885640 1 -- 172.21.15.148:0/605088845 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0006db90 con 0x7f4a0006d3b0 2024-03-04T12:12:31.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.640+0000 7f4a05887640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:31.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.641+0000 7f4a04885640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0006d3b0 0x7f4a0015c120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a000b28a0 tx=0x7f49fc02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fcfa62e54112d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.641+0000 7f49f7fff640 1 -- 172.21.15.148:0/605088845 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc031770 con 0x7f4a0006d3b0 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.641+0000 7f49f7fff640 1 -- 172.21.15.148:0/605088845 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc037070 con 0x7f4a0006d3b0 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.641+0000 7f49f7fff640 1 -- 172.21.15.148:0/605088845 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc031b30 con 0x7f4a0006d3b0 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 -- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0006d3b0 msgr2=0x7f4a0015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0006d3b0 0x7f4a0015c120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a000b28a0 tx=0x7f49fc02b810 comp rx=0 tx=0).stop 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 -- 172.21.15.148:0/605088845 shutdown_connections 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0015b410 0x7f4a0015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 --2- 172.21.15.148:0/605088845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0006d3b0 0x7f4a0015c120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 -- 172.21.15.148:0/605088845 >> 172.21.15.148:0/605088845 conn(0x7f4a0006acd0 msgr2=0x7f4a0006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:31.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 -- 172.21.15.148:0/605088845 shutdown_connections 2024-03-04T12:12:31.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.642+0000 7f4a072ea640 1 -- 172.21.15.148:0/605088845 wait complete. 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 Processor -- start 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 -- start start 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0015b410 0x7f4a000cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a000cf3a0 0x7f4a000ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006e690 con 0x7f4a0015b030 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.643+0000 7f4a072ea640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006e150 con 0x7f4a0015b410 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a072ea640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006e810 con 0x7f4a000cf3a0 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45626/0 (socket says 172.21.15.148:45626) 2024-03-04T12:12:31.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 -- 172.21.15.148:0/1894934470 learned_addr learned my addr 172.21.15.148:0/1894934470 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05887640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a000cf3a0 0x7f4a000ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a04885640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0015b410 0x7f4a000cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 -- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a000cf3a0 msgr2=0x7f4a000ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a000cf3a0 0x7f4a000ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 -- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0015b410 msgr2=0x7f4a000cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0015b410 0x7f4a000cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 -- 172.21.15.148:0/1894934470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0014fc40 con 0x7f4a0015b030 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a05086640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c400 tx=0x7f49f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0017070 con 0x7f4a0015b030 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0005ce0 con 0x7f4a0015b030 2024-03-04T12:12:31.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0014ee80 con 0x7f4a0015b030 2024-03-04T12:12:31.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.644+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00072f0 con 0x7f4a0015b030 2024-03-04T12:12:31.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.645+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0014f440 con 0x7f4a0015b030 2024-03-04T12:12:31.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.646+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000cf7b0 con 0x7f4a0015b030 2024-03-04T12:12:31.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.646+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f49f001a070 con 0x7f4a0015b030 2024-03-04T12:12:31.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.646+0000 7f49f5ffb640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:31.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.646+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49f0089810 con 0x7f4a0015b030 2024-03-04T12:12:31.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.646+0000 7f4a04885640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:31.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.647+0000 7f4a04885640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 0x7f49e406d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f49fc036000 tx=0x7f49fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:31.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.651+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a000cf7b0 con 0x7f4a0015b030 2024-03-04T12:12:31.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.858+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a00074f60 con 0x7f4a0015b030 2024-03-04T12:12:31.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.858+0000 7f49f5ffb640 1 -- 172.21.15.148:0/1894934470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f49f0087e20 con 0x7f4a0015b030 2024-03-04T12:12:31.860 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 msgr2=0x7f49e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 0x7f49e406d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f49fc036000 tx=0x7f49fc008030 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 msgr2=0x7f4a0014c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c400 tx=0x7f49f000c800 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 shutdown_connections 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a000cf3a0 0x7f4a000ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0015b410 0x7f4a000cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 --2- 172.21.15.148:0/1894934470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0015b030 0x7f4a0014c2e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 >> 172.21.15.148:0/1894934470 conn(0x7f4a0006acd0 msgr2=0x7f4a0006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 shutdown_connections 2024-03-04T12:12:31.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:31.860+0000 7f4a072ea640 1 -- 172.21.15.148:0/1894934470 wait complete. 2024-03-04T12:12:31.872 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T12:12:31.872 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:12:31.872 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-04T12:12:31.872 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-04T12:12:31.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-04T12:12:31.873 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T12:12:31.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T12:12:31.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:12:31.873 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-04T12:12:31.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.873 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-04T12:12:31.892 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42696sec preferred_lft 42696sec 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-03-04T12:12:31.893 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:31.893 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T12:12:31.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T12:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:32.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.157 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:12:32.185 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-04T12:12:32.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.187 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:12:32.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.263 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:32.263 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:32.263 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:12:32.264 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:12:32.264 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:12:32.264 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:32.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.525 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-04T12:12:32.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.541 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-03-04T12:12:32.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.595 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:32.595 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:32.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T12:12:32.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-04T12:12:32.595 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:32.677 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-04T12:12:32.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-04T12:12:32.678 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:32.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.818 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:32.818 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:32.818 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-04T12:12:32.818 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T12:12:32.818 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:32.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:32.859 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-04T12:12:32.915 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-04T12:12:32.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:32.915 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-04T12:12:32.969 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-04T12:12:32.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:32.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:33.054 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:33.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:33.161 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:33.162 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:33.162 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-04T12:12:33.162 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@d4b3ed31-a94c-42d0-968e-1d6a4e199a28.cephfs=/ 2024-03-04T12:12:33.162 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-04T12:12:33.163 INFO:teuthology.orchestra.run.smithi191.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-04T12:12:33.163 INFO:teuthology.orchestra.run.smithi191.stdout: You just mounted a file system that supports labels which does not 2024-03-04T12:12:33.163 INFO:teuthology.orchestra.run.smithi191.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T12:12:33.163 INFO:teuthology.orchestra.run.smithi191.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T12:12:33.163 INFO:teuthology.orchestra.run.smithi191.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T12:12:33.164 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T12:12:33.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-04T12:12:33.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.201 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:33.257 INFO:teuthology.orchestra.run.smithi191.stdout:client4574 2024-03-04T12:12:33.258 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:33.258 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:33.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.313 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:33.368 INFO:teuthology.orchestra.run.smithi191.stdout:client4574 2024-03-04T12:12:33.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.368 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:33.424 INFO:teuthology.orchestra.run.smithi191.stdout:client4574 2024-03-04T12:12:33.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.424 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d4b3ed31-a94c-42d0-968e-1d6a4e199a28.client4574/status') 2024-03-04T12:12:33.491 INFO:teuthology.orchestra.run.smithi191.stdout:instance: client.4574 (3)192.168.0.1:0/3916713441 2024-03-04T12:12:33.491 INFO:teuthology.orchestra.run.smithi191.stdout:blocklisted: false 2024-03-04T12:12:33.492 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records in 2024-03-04T12:12:33.492 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records out 2024-03-04T12:12:33.492 INFO:teuthology.orchestra.run.smithi191.stderr:69 bytes copied, 1.9759e-05 s, 3.5 MB/s 2024-03-04T12:12:33.492 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-04T12:12:33.492 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T12:12:33.492 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-04T12:12:33.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:12:33.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.602+0000 7fbee873b640 1 Processor -- start 2024-03-04T12:12:33.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.603+0000 7fbee873b640 1 -- start start 2024-03-04T12:12:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.603+0000 7fbee873b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 0x7fbee00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.603+0000 7fbee873b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbee00b5470 0x7fbee00b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee873b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee015e1e0 con 0x7fbee00b9380 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee873b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00d2820 con 0x7fbee00b5470 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee873b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00d1c10 con 0x7fbee015de00 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee5cd6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbee00b9380 0x7fbee00b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38228/0 (socket says 172.21.15.148:38228) 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee5cd6640 1 -- 172.21.15.148:0/1369258580 learned_addr learned my addr 172.21.15.148:0/1369258580 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee6cd8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbee00b5470 0x7fbee00b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee64d7640 1 --2- 172.21.15.148:0/1369258580 >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 0x7fbee00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee6cd8640 1 -- 172.21.15.148:0/1369258580 >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 msgr2=0x7fbee00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee6cd8640 1 --2- 172.21.15.148:0/1369258580 >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 0x7fbee00b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee6cd8640 1 -- 172.21.15.148:0/1369258580 >> v1:172.21.15.148:6789/0 conn(0x7fbee00b9380 legacy=0x7fbee00b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee6cd8640 1 -- 172.21.15.148:0/1369258580 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00d1ef0 con 0x7fbee00b5470 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.604+0000 7fbee64d7640 1 --2- 172.21.15.148:0/1369258580 >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 0x7fbee00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee6cd8640 1 --2- 172.21.15.148:0/1369258580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b5470 0x7fbee00b5900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce240 tx=0x7fbedc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c6c73ac3a3357 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee54d5640 1 -- 172.21.15.148:0/1369258580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc035020 con 0x7fbee00b5470 2024-03-04T12:12:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee54d5640 1 -- 172.21.15.148:0/1369258580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc031830 con 0x7fbee00b5470 2024-03-04T12:12:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee54d5640 1 -- 172.21.15.148:0/1369258580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc031c00 con 0x7fbee00b5470 2024-03-04T12:12:33.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee873b640 1 -- 172.21.15.148:0/1369258580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b5470 msgr2=0x7fbee00b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.605+0000 7fbee873b640 1 --2- 172.21.15.148:0/1369258580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b5470 0x7fbee00b5900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce240 tx=0x7fbedc02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 -- 172.21.15.148:0/1369258580 shutdown_connections 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 --2- 172.21.15.148:0/1369258580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b5470 0x7fbee00b5900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 --2- 172.21.15.148:0/1369258580 >> v2:172.21.15.191:3300/0 conn(0x7fbee015de00 0x7fbee00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 -- 172.21.15.148:0/1369258580 >> 172.21.15.148:0/1369258580 conn(0x7fbee00d0470 msgr2=0x7fbee00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 -- 172.21.15.148:0/1369258580 shutdown_connections 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.606+0000 7fbee873b640 1 -- 172.21.15.148:0/1369258580 wait complete. 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.607+0000 7fbee873b640 1 Processor -- start 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.607+0000 7fbee873b640 1 -- start start 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.607+0000 7fbee873b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 0x7fbee00ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 0x7fbee014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b97e0 con 0x7fbee00b9380 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00d24b0 con 0x7fbee00b5470 2024-03-04T12:12:33.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b72a0 con 0x7fbee015de00 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58962/0 (socket says 172.21.15.148:58962) 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 -- 172.21.15.148:0/3621397222 learned_addr learned my addr 172.21.15.148:0/3621397222 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee64d7640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 0x7fbee00ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee6cd8640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 0x7fbee014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 -- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 msgr2=0x7fbee014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 0x7fbee014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 -- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 msgr2=0x7fbee00ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 0x7fbee00ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 -- 172.21.15.148:0/3621397222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee014d5d0 con 0x7fbee00b9380 2024-03-04T12:12:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee64d7640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 0x7fbee00ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee6cd8640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 0x7fbee014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee5cd6640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c430 tx=0x7fbed000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0017070 con 0x7fbee00b9380 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0005ce0 con 0x7fbee00b9380 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee014d8b0 con 0x7fbee00b9380 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00072f0 con 0x7fbee00b9380 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.608+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee014afe0 con 0x7fbee00b9380 2024-03-04T12:12:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.609+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee0074f60 con 0x7fbee00b9380 2024-03-04T12:12:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.610+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbed001a070 con 0x7fbee00b9380 2024-03-04T12:12:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.610+0000 7fbed6ffd640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.610+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbed0089960 con 0x7fbee00b9380 2024-03-04T12:12:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.610+0000 7fbee64d7640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.611+0000 7fbee64d7640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbed800e550 tx=0x7fbed8007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:33.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.615+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbed00584a0 con 0x7fbee00b9380 2024-03-04T12:12:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.821+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbee00cdee0 con 0x7fbee00b9380 2024-03-04T12:12:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.822+0000 7fbed6ffd640 1 -- 172.21.15.148:0/3621397222 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbed0057da0 con 0x7fbee00b9380 2024-03-04T12:12:33.823 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T12:12:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.823+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 msgr2=0x7fbec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.823+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbed800e550 tx=0x7fbed8007ab0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 msgr2=0x7fbee014cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c430 tx=0x7fbed000c8f0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 shutdown_connections 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee015de00 0x7fbee014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee00b9380 0x7fbee014cf10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 --2- 172.21.15.148:0/3621397222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee00b5470 0x7fbee00ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 >> 172.21.15.148:0/3621397222 conn(0x7fbee00d0470 msgr2=0x7fbee00b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 shutdown_connections 2024-03-04T12:12:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:33.824+0000 7fbee873b640 1 -- 172.21.15.148:0/3621397222 wait complete. 2024-03-04T12:12:33.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T12:12:33.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:12:33.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-04T12:12:33.836 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-04T12:12:33.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-04T12:12:33.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T12:12:33.837 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T12:12:33.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:12:33.837 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-04T12:12:33.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.837 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:33.853 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42694sec preferred_lft 42694sec 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:33.854 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether d2:52:f6:3f:9d:3c brd ff:ff:ff:ff:ff:ff 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::d0a4:a9ff:fe3d:32e5/64 scope link 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether d2:52:f6:3f:9d:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::d052:f6ff:fe3f:9d3c/64 scope link tentative 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:33.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.855 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:12:33.919 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-04T12:12:33.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.921 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-04T12:12:33.976 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-04T12:12:33.976 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:12:33.976 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:12:33.976 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:12:33.976 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:12:33.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:12:33.977 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:34.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.072 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-04T12:12:34.086 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-04T12:12:34.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.087 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-03-04T12:12:34.141 INFO:teuthology.orchestra.run.smithi191.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-04T12:12:34.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.141 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:34.141 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:34.141 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-04T12:12:34.141 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-04T12:12:34.141 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:34.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.291 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:34.292 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 6a:15:b7:2e:78:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-04T12:12:34.292 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-04T12:12:34.292 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.292 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::6815:b7ff:fe2e:782d/64 scope link tentative 2024-03-04T12:12:34.292 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 6a:15:b7:2e:78:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-04T12:12:34.360 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-04T12:12:34.361 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.361 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::6815:b7ff:fe2e:782d/64 scope link tentative 2024-03-04T12:12:34.361 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:12:34.362 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-04T12:12:34.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-04T12:12:34.362 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:34.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.488 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-04T12:12:34.488 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-04T12:12:34.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.1 up 2024-03-04T12:12:34.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.1 master ceph-brx 2024-03-04T12:12:34.488 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-04T12:12:34.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:34.530 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-04T12:12:34.584 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-04T12:12:34.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:12:34.584 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-04T12:12:34.639 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-04T12:12:34.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.639 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:34.723 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:34.724 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:34.730 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:12:34.730 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 3@d4b3ed31-a94c-42d0-968e-1d6a4e199a28.cephfs=/ 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.148:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout: You just mounted a file system that supports labels which does not 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T12:12:34.731 INFO:teuthology.orchestra.run.smithi191.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T12:12:34.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T12:12:34.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.733 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-04T12:12:34.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.760 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:34.816 INFO:teuthology.orchestra.run.smithi191.stdout:client4580 2024-03-04T12:12:34.816 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:34.816 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:34.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.872 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:34.927 INFO:teuthology.orchestra.run.smithi191.stdout:client4580 2024-03-04T12:12:34.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.927 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:12:34.983 INFO:teuthology.orchestra.run.smithi191.stdout:client4580 2024-03-04T12:12:34.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:34.983 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d4b3ed31-a94c-42d0-968e-1d6a4e199a28.client4580/status') 2024-03-04T12:12:35.050 INFO:teuthology.orchestra.run.smithi191.stdout:instance: client.4580 (3)192.168.0.2:0/344754314 2024-03-04T12:12:35.050 INFO:teuthology.orchestra.run.smithi191.stdout:blocklisted: false 2024-03-04T12:12:35.050 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records in 2024-03-04T12:12:35.050 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records out 2024-03-04T12:12:35.051 INFO:teuthology.orchestra.run.smithi191.stderr:68 bytes copied, 2.005e-05 s, 3.4 MB/s 2024-03-04T12:12:35.052 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-04T12:12:35.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.180+0000 7f2284e43640 1 Processor -- start 2024-03-04T12:12:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.180+0000 7f2284e43640 1 -- start start 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f2284e43640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f228015b030 0x7f228015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f2284e43640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228015b410 0x7f228015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f2284e43640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228015be00 con 0x7f228006d3b0 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f2284e43640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22800b2720 con 0x7f228015b030 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f2284e43640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228006d8b0 con 0x7f228015b410 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f228015b030 0x7f228015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227dd9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f228006d3b0 0x7f228015dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38242/0 (socket says 172.21.15.148:38242) 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227dd9b640 1 -- 172.21.15.148:0/3428552280 learned_addr learned my addr 172.21.15.148:0/3428552280 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227ed9d640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228015b410 0x7f228015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227e59c640 1 -- 172.21.15.148:0/3428552280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228015b410 msgr2=0x7f228015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227e59c640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228015b410 0x7f228015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227e59c640 1 -- 172.21.15.148:0/3428552280 >> v1:172.21.15.148:6789/0 conn(0x7f228006d3b0 legacy=0x7f228015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227d59a640 1 -- 172.21.15.148:0/3428552280 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104411813 0 0) 0x7f228015be00 con 0x7f228006d3b0 2024-03-04T12:12:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.181+0000 7f227e59c640 1 -- 172.21.15.148:0/3428552280 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228006db90 con 0x7f228015b030 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.182+0000 7f227e59c640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f228015c120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22800b28a0 tx=0x7f227002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4c74326d91256e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.182+0000 7f227d59a640 1 -- 172.21.15.148:0/3428552280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270039070 con 0x7f228015b030 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.182+0000 7f227d59a640 1 -- 172.21.15.148:0/3428552280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227002cbb0 con 0x7f228015b030 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.182+0000 7f227d59a640 1 -- 172.21.15.148:0/3428552280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270034440 con 0x7f228015b030 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 -- 172.21.15.148:0/3428552280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 msgr2=0x7f228015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f228015c120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22800b28a0 tx=0x7f227002be00 comp rx=0 tx=0).stop 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 -- 172.21.15.148:0/3428552280 shutdown_connections 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228015b410 0x7f228015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 --2- 172.21.15.148:0/3428552280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f228015c120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 -- 172.21.15.148:0/3428552280 >> 172.21.15.148:0/3428552280 conn(0x7f228006acd0 msgr2=0x7f228006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 -- 172.21.15.148:0/3428552280 shutdown_connections 2024-03-04T12:12:35.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.183+0000 7f2284e43640 1 -- 172.21.15.148:0/3428552280 wait complete. 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.184+0000 7f2284e43640 1 Processor -- start 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.184+0000 7f2284e43640 1 -- start start 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228006d3b0 0x7f22801596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 0x7f228015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228006e690 con 0x7f228015b030 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228006e150 con 0x7f228015b410 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f2284e43640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228006e810 con 0x7f228006d3b0 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58986/0 (socket says 172.21.15.148:58986) 2024-03-04T12:12:35.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 -- 172.21.15.148:0/3023066705 learned_addr learned my addr 172.21.15.148:0/3023066705 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227e59c640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228006d3b0 0x7f22801596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227ed9d640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 0x7f228015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 -- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228006d3b0 msgr2=0x7f22801596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228006d3b0 0x7f22801596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 -- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 msgr2=0x7f228015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 0x7f228015d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227dd9b640 1 -- 172.21.15.148:0/3023066705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280148a50 con 0x7f228015b030 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.185+0000 7f227ed9d640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 0x7f228015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f227dd9b640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f227400c4b0 tx=0x7f227400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274018070 con 0x7f228015b030 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274007500 con 0x7f228015b030 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2280149010 con 0x7f228015b030 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274004a20 con 0x7f228015b030 2024-03-04T12:12:35.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.186+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22801495d0 con 0x7f228015b030 2024-03-04T12:12:35.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.187+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2280149910 con 0x7f228015b030 2024-03-04T12:12:35.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.187+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f227401b070 con 0x7f228015b030 2024-03-04T12:12:35.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.187+0000 7f226effd640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.187+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2274089960 con 0x7f228015b030 2024-03-04T12:12:35.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.188+0000 7f227e59c640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.188+0000 7f227e59c640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 0x7f226006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2270036000 tx=0x7f2270030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.192+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2280149910 con 0x7f228015b030 2024-03-04T12:12:35.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.363+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2280074f60 con 0x7f228015b030 2024-03-04T12:12:35.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.364+0000 7f226effd640 1 -- 172.21.15.148:0/3023066705 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f22740584a0 con 0x7f228015b030 2024-03-04T12:12:35.366 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:35.366 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:35.010568+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 msgr2=0x7f226006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 0x7f226006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2270036000 tx=0x7f2270030000 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 msgr2=0x7f22801e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f227400c4b0 tx=0x7f227400c970 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 shutdown_connections 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f228015b410 0x7f228015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228015b030 0x7f22801e6260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 --2- 172.21.15.148:0/3023066705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f228006d3b0 0x7f22801596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 >> 172.21.15.148:0/3023066705 conn(0x7f228006acd0 msgr2=0x7f228006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 shutdown_connections 2024-03-04T12:12:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.366+0000 7f2284e43640 1 -- 172.21.15.148:0/3023066705 wait complete. 2024-03-04T12:12:35.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:35.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.483+0000 7fea40a8c640 1 Processor -- start 2024-03-04T12:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.483+0000 7fea40a8c640 1 -- start start 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.483+0000 7fea40a8c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea3c0cf660 0x7fea3c0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea40a8c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c162450 con 0x7fea3c1616e0 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea40a8c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0cc2d0 con 0x7fea3c15da20 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea40a8c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c0d0000 con 0x7fea3c0cf660 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea3ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fea3c1616e0 0x7fea3c161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38254/0 (socket says 172.21.15.148:38254) 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea3ad9d640 1 -- 172.21.15.148:0/1869666207 learned_addr learned my addr 172.21.15.148:0/1869666207 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea39d9b640 1 --2- 172.21.15.148:0/1869666207 >> v2:172.21.15.191:3300/0 conn(0x7fea3c0cf660 0x7fea3c0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264388604 0 0) 0x7fea3c162450 con 0x7fea3c1616e0 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea24003520 con 0x7fea3c1616e0 2024-03-04T12:12:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea39d9b640 1 -- 172.21.15.148:0/1869666207 >> v1:172.21.15.191:6789/0 conn(0x7fea3c15da20 legacy=0x7fea3c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451539278 0 0) 0x7fea3c0cc2d0 con 0x7fea3c15da20 2024-03-04T12:12:35.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea39d9b640 1 -- 172.21.15.148:0/1869666207 >> v1:172.21.15.148:6789/0 conn(0x7fea3c1616e0 legacy=0x7fea3c161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.484+0000 7fea39d9b640 1 -- 172.21.15.148:0/1869666207 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0cc5b0 con 0x7fea3c0cf660 2024-03-04T12:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.485+0000 7fea39d9b640 1 --2- 172.21.15.148:0/1869666207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c0cfac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea300097a0 tx=0x7fea3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38a57f751042b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.485+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30035020 con 0x7fea3c0cf660 2024-03-04T12:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.485+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30031830 con 0x7fea3c0cf660 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.485+0000 7fea3959a640 1 -- 172.21.15.148:0/1869666207 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30031be0 con 0x7fea3c0cf660 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 -- 172.21.15.148:0/1869666207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 msgr2=0x7fea3c0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 --2- 172.21.15.148:0/1869666207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c0cfac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea300097a0 tx=0x7fea3002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 -- 172.21.15.148:0/1869666207 shutdown_connections 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 --2- 172.21.15.148:0/1869666207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c0cfac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 -- 172.21.15.148:0/1869666207 >> 172.21.15.148:0/1869666207 conn(0x7fea3c0b33a0 msgr2=0x7fea3c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 -- 172.21.15.148:0/1869666207 shutdown_connections 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.486+0000 7fea40a8c640 1 -- 172.21.15.148:0/1869666207 wait complete. 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.487+0000 7fea40a8c640 1 Processor -- start 2024-03-04T12:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.487+0000 7fea40a8c640 1 -- start start 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3c15da20 0x7fea3c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15de00 con 0x7fea3c1616e0 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0cc920 con 0x7fea3c0cf660 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea40a8c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c06bde0 con 0x7fea3c15da20 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59000/0 (socket says 172.21.15.148:59000) 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 -- 172.21.15.148:0/341291382 learned_addr learned my addr 172.21.15.148:0/341291382 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3a59c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea39d9b640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3c15da20 0x7fea3c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 -- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3c15da20 msgr2=0x7fea3c0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3c15da20 0x7fea3c0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 -- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 msgr2=0x7fea3c14fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c14fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 -- 172.21.15.148:0/341291382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0ad180 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.488+0000 7fea3ad9d640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ca20 tx=0x7fea3400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.489+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34017070 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.489+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3400abd0 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.489+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c0ad460 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.489+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34012400 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.489+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c0acdc0 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.490+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c074f60 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.490+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fea3401a070 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.490+0000 7fea2affd640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.490+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fea3408a230 con 0x7fea3c1616e0 2024-03-04T12:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.491+0000 7fea3a59c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.491+0000 7fea3a59c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 0x7fea1c06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000f60 tx=0x7fea2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.495+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea340586f0 con 0x7fea3c1616e0 2024-03-04T12:12:35.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.666+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea3c0ce770 con 0x7fea3c1616e0 2024-03-04T12:12:35.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.666+0000 7fea2affd640 1 -- 172.21.15.148:0/341291382 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea3405c790 con 0x7fea3c1616e0 2024-03-04T12:12:35.668 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 msgr2=0x7fea1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 0x7fea1c06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000f60 tx=0x7fea2c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 msgr2=0x7fea3c0ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ca20 tx=0x7fea3400cee0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 shutdown_connections 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3c1616e0 0x7fea3c0ae080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3c15da20 0x7fea3c0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 --2- 172.21.15.148:0/341291382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea3c0cf660 0x7fea3c14fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 >> 172.21.15.148:0/341291382 conn(0x7fea3c0b33a0 msgr2=0x7fea3c0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.668+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 shutdown_connections 2024-03-04T12:12:35.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.669+0000 7fea40a8c640 1 -- 172.21.15.148:0/341291382 wait complete. 2024-03-04T12:12:35.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:35.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.783+0000 7f1f5cadc640 1 Processor -- start 2024-03-04T12:12:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.783+0000 7f1f5cadc640 1 -- start start 2024-03-04T12:12:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5cadc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 0x7f1f580b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5cadc640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f580d1e90 0x7f1f580cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5cadc640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5815bf50 con 0x7f1f5815f660 2024-03-04T12:12:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5cadc640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f580cf890 con 0x7f1f580d1e90 2024-03-04T12:12:35.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5cadc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5815fb60 con 0x7f1f5815bb70 2024-03-04T12:12:35.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 0x7f1f580b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f56d9d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f580d1e90 0x7f1f580cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f55d9b640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1f5815f660 0x7f1f580d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:38454/0 (socket says 172.21.15.148:38454) 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 0x7f1f580b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:52212/0 (socket says 172.21.15.148:52212) 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f56d9d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f580d1e90 0x7f1f580cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:38762/0 (socket says 172.21.15.148:38762) 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.784+0000 7f1f5659c640 1 -- 172.21.15.148:0/1466163036 learned_addr learned my addr 172.21.15.148:0/1466163036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.785+0000 7f1f56d9d640 1 -- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 msgr2=0x7f1f580b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.785+0000 7f1f56d9d640 1 --2- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 0x7f1f580b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.785+0000 7f1f56d9d640 1 -- 172.21.15.148:0/1466163036 >> v1:172.21.15.191:6789/0 conn(0x7f1f5815f660 legacy=0x7f1f580d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.785+0000 7f1f56d9d640 1 -- 172.21.15.148:0/1466163036 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5815fe40 con 0x7f1f580d1e90 2024-03-04T12:12:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.785+0000 7f1f56d9d640 1 --2- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580cfd20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009d90 tx=0x7f1f5002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e580adddb02e5813 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5559a640 1 -- 172.21.15.148:0/1466163036 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50039070 con 0x7f1f580d1e90 2024-03-04T12:12:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5559a640 1 -- 172.21.15.148:0/1466163036 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5002cbc0 con 0x7f1f580d1e90 2024-03-04T12:12:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5559a640 1 -- 172.21.15.148:0/1466163036 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50034440 con 0x7f1f580d1e90 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 -- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 msgr2=0x7f1f580cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580cfd20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009d90 tx=0x7f1f5002be10 comp rx=0 tx=0).stop 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 -- 172.21.15.148:0/1466163036 shutdown_connections 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580cfd20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/1466163036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815bb70 0x7f1f580b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 -- 172.21.15.148:0/1466163036 >> 172.21.15.148:0/1466163036 conn(0x7f1f580b33a0 msgr2=0x7f1f580b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 -- 172.21.15.148:0/1466163036 shutdown_connections 2024-03-04T12:12:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.786+0000 7f1f5cadc640 1 -- 172.21.15.148:0/1466163036 wait complete. 2024-03-04T12:12:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 Processor -- start 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 -- start start 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815f660 0x7f1f58158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580b0f80 con 0x7f1f5815bb70 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58160400 con 0x7f1f580d1e90 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.788+0000 7f1f5cadc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580b1100 con 0x7f1f5815f660 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59012/0 (socket says 172.21.15.148:59012) 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 -- 172.21.15.148:0/3117336180 learned_addr learned my addr 172.21.15.148:0/3117336180 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f56d9d640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815f660 0x7f1f58158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f5659c640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 -- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815f660 msgr2=0x7f1f58158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815f660 0x7f1f58158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 -- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 msgr2=0x7f1f580c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 -- 172.21.15.148:0/3117336180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580ca260 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f55d9b640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c3c0 tx=0x7f1f4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0074e0 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c007680 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f58156cb0 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004a00 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.789+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f580c8f60 con 0x7f1f5815bb70 2024-03-04T12:12:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.790+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f58074f60 con 0x7f1f5815bb70 2024-03-04T12:12:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.791+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1f4c01a020 con 0x7f1f5815bb70 2024-03-04T12:12:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.791+0000 7f1f46ffd640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 0x7f1f3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.791+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1f4c089b40 con 0x7f1f5815bb70 2024-03-04T12:12:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.791+0000 7f1f5659c640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 0x7f1f3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.792+0000 7f1f5659c640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 0x7f1f3806d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009bd0 tx=0x7f1f48009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:35.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.795+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f4c058000 con 0x7f1f5815bb70 2024-03-04T12:12:35.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.967+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f5806e620 con 0x7f1f5815bb70 2024-03-04T12:12:35.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.967+0000 7f1f46ffd640 1 -- 172.21.15.148:0/3117336180 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f4c05c0a0 con 0x7f1f5815bb70 2024-03-04T12:12:35.969 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 msgr2=0x7f1f3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 0x7f1f3806d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009bd0 tx=0x7f1f48009240 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 msgr2=0x7f1f58156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c3c0 tx=0x7f1f4c00c880 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 shutdown_connections 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1f3806b500 0x7f1f3806d9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f5815f660 0x7f1f58158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f5815bb70 0x7f1f58156770 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 --2- 172.21.15.148:0/3117336180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f580d1e90 0x7f1f580c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 >> 172.21.15.148:0/3117336180 conn(0x7f1f580b33a0 msgr2=0x7f1f580b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 shutdown_connections 2024-03-04T12:12:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:35.969+0000 7f1f5cadc640 1 -- 172.21.15.148:0/3117336180 wait complete. 2024-03-04T12:12:35.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:36.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.086+0000 7fc4b968e640 1 Processor -- start 2024-03-04T12:12:36.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.086+0000 7fc4b968e640 1 -- start start 2024-03-04T12:12:36.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b968e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc4b40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b968e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b415ca50 con 0x7fc4b415c2e0 2024-03-04T12:12:36.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b968e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b415cd30 con 0x7fc4b40ceec0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b968e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b415d2f0 con 0x7fc4b40ccbd0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b27fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc4b415c2e0 0x7fc4b40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38260/0 (socket says 172.21.15.148:38260) 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.087+0000 7fc4b27fc640 1 -- 172.21.15.148:0/101958853 learned_addr learned my addr 172.21.15.148:0/101958853 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b2ffd640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc4b40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624899241 0 0) 0x7fc4b415ca50 con 0x7fc4b415c2e0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc49c003520 con 0x7fc4b415c2e0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2019687258 0 0) 0x7fc49c003520 con 0x7fc4b415c2e0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 msgr2=0x7fc4b40b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc4b40b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 >> v1:172.21.15.191:6789/0 conn(0x7fc4b40ceec0 legacy=0x7fc4b40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b415d010 con 0x7fc4b415c2e0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fc4a8001670 con 0x7fc4b415c2e0 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 >> v1:172.21.15.148:6789/0 conn(0x7fc4b415c2e0 legacy=0x7fc4b40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc49c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc49c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c00b9f0 con 0x7fc49c003f20 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c00bba0 con 0x7fc49c004d40 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c00bda0 con 0x7fc4b40ccbd0 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc49c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b2ffd640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc49c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b37fe640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 -- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 msgr2=0x7fc49c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b40ccbd0 0x7fc49c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 -- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc49c004d40 msgr2=0x7fc49c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.088+0000 7fc4b27fc640 1 -- 172.21.15.148:0/101958853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c003520 con 0x7fc49c003f20 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.089+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc49c004800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c190 tx=0x7fc49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64bae91ec37f462 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.089+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007cc0 con 0x7fc49c003f20 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.089+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a8030070 con 0x7fc49c003f20 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.089+0000 7fc4b1ffb640 1 -- 172.21.15.148:0/101958853 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8005d50 con 0x7fc49c003f20 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 -- 172.21.15.148:0/101958853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 msgr2=0x7fc49c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 --2- 172.21.15.148:0/101958853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc49c004800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c190 tx=0x7fc49c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b37fe640 1 -- 172.21.15.148:0/101958853 reap_dead start 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 -- 172.21.15.148:0/101958853 shutdown_connections 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 -- 172.21.15.148:0/101958853 >> 172.21.15.148:0/101958853 conn(0x7fc4b4066dc0 msgr2=0x7fc4b40671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 -- 172.21.15.148:0/101958853 shutdown_connections 2024-03-04T12:12:36.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.090+0000 7fc4b968e640 1 -- 172.21.15.148:0/101958853 wait complete. 2024-03-04T12:12:36.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.091+0000 7fc4b968e640 1 Processor -- start 2024-03-04T12:12:36.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.091+0000 7fc4b968e640 1 -- start start 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b41554a0 0x7fc4b415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b4154f10 0x7fc4b40caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40ccfb0 con 0x7fc49c003f20 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b415d6d0 con 0x7fc4b4154f10 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b968e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b415eb70 con 0x7fc4b41554a0 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59026/0 (socket says 172.21.15.148:59026) 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 -- 172.21.15.148:0/2563396305 learned_addr learned my addr 172.21.15.148:0/2563396305 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b41554a0 0x7fc4b415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b37fe640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b4154f10 0x7fc4b40caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 -- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b41554a0 msgr2=0x7fc4b415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b41554a0 0x7fc4b415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 -- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b4154f10 msgr2=0x7fc4b40caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b4154f10 0x7fc4b40caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 -- 172.21.15.148:0/2563396305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40c6460 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.092+0000 7fc4b2ffd640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4011e60 tx=0x7fc4a400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.093+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4015630 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.093+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40157d0 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.093+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b40c97f0 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.093+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4015ba0 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.093+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b40c66d0 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.094+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b4074f60 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.094+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc4a401d400 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.094+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.094+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc4a408b770 con 0x7fc49c003f20 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.094+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.095+0000 7fc4b27fc640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 0x7fc4a006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8034000 tx=0x7fc4a8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.099+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4a4019030 con 0x7fc49c003f20 2024-03-04T12:12:36.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.268+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4b40c8bc0 con 0x7fc49c003f20 2024-03-04T12:12:36.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.268+0000 7fc49b7fe640 1 -- 172.21.15.148:0/2563396305 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4a4010040 con 0x7fc49c003f20 2024-03-04T12:12:36.270 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 msgr2=0x7fc4a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 0x7fc4a006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8034000 tx=0x7fc4a8009030 comp rx=0 tx=0).stop 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 msgr2=0x7fc4b41576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4011e60 tx=0x7fc4a400f580 comp rx=0 tx=0).stop 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 shutdown_connections 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b4154f10 0x7fc4b40caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4b41554a0 0x7fc4b415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 --2- 172.21.15.148:0/2563396305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c003f20 0x7fc4b41576a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 >> 172.21.15.148:0/2563396305 conn(0x7fc4b4066dc0 msgr2=0x7fc4b40cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 shutdown_connections 2024-03-04T12:12:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.270+0000 7fc4b968e640 1 -- 172.21.15.148:0/2563396305 wait complete. 2024-03-04T12:12:36.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:36.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.385+0000 7f7dfaf59640 1 Processor -- start 2024-03-04T12:12:36.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 -- start start 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7df415df00 0x7f7df406b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7df40cfa40 0x7f7df40cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df40701d0 con 0x7f7df406fa80 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df40704b0 con 0x7f7df415df00 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7dfaf59640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df40cc8e0 con 0x7f7df40cfa40 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7df8cf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7df415df00 0x7f7df406b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7df3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7df406fa80 0x7f7df40cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38262/0 (socket says 172.21.15.148:38262) 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.386+0000 7f7df3fff640 1 -- 172.21.15.148:0/801559178 learned_addr learned my addr 172.21.15.148:0/801559178 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:36.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df94f6640 1 --2- 172.21.15.148:0/801559178 >> v2:172.21.15.191:3300/0 conn(0x7f7df40cfa40 0x7f7df40cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df8cf5640 1 -- 172.21.15.148:0/801559178 >> v2:172.21.15.191:3300/0 conn(0x7f7df40cfa40 msgr2=0x7f7df40cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df8cf5640 1 --2- 172.21.15.148:0/801559178 >> v2:172.21.15.191:3300/0 conn(0x7f7df40cfa40 0x7f7df40cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df8cf5640 1 -- 172.21.15.148:0/801559178 >> v1:172.21.15.148:6789/0 conn(0x7f7df406fa80 legacy=0x7f7df40cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df8cf5640 1 -- 172.21.15.148:0/801559178 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df40ccbc0 con 0x7f7df415df00 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.387+0000 7f7df8cf5640 1 --2- 172.21.15.148:0/801559178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df415df00 0x7f7df406b700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7df40ce230 tx=0x7f7de802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a107943b6f9581 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7df37fe640 1 -- 172.21.15.148:0/801559178 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8037070 con 0x7f7df415df00 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7df37fe640 1 -- 172.21.15.148:0/801559178 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de8030710 con 0x7f7df415df00 2024-03-04T12:12:36.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7df37fe640 1 -- 172.21.15.148:0/801559178 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8030ae0 con 0x7f7df415df00 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 -- 172.21.15.148:0/801559178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df415df00 msgr2=0x7f7df406b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/801559178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df415df00 0x7f7df406b700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7df40ce230 tx=0x7f7de802b710 comp rx=0 tx=0).stop 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 -- 172.21.15.148:0/801559178 shutdown_connections 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/801559178 >> v2:172.21.15.191:3300/0 conn(0x7f7df40cfa40 0x7f7df40cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/801559178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df415df00 0x7f7df406b700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 -- 172.21.15.148:0/801559178 >> 172.21.15.148:0/801559178 conn(0x7f7df40b2250 msgr2=0x7f7df40b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 -- 172.21.15.148:0/801559178 shutdown_connections 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.388+0000 7f7dfaf59640 1 -- 172.21.15.148:0/801559178 wait complete. 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.389+0000 7f7dfaf59640 1 Processor -- start 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.389+0000 7f7dfaf59640 1 -- start start 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df406fa80 0x7f7df414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df415df00 0x7f7df41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40d0410 con 0x7f7df40cfa40 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df406fee0 con 0x7f7df406fa80 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7dfaf59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40cd3c0 con 0x7f7df415df00 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59044/0 (socket says 172.21.15.148:59044) 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7df3fff640 1 -- 172.21.15.148:0/757335816 learned_addr learned my addr 172.21.15.148:0/757335816 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7df94f6640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df415df00 0x7f7df41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.390+0000 7f7df8cf5640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df406fa80 0x7f7df414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 -- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df415df00 msgr2=0x7f7df41563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df415df00 0x7f7df41563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 -- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df406fa80 msgr2=0x7f7df414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df406fa80 0x7f7df414ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 -- 172.21.15.148:0/757335816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4155330 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df3fff640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c800 tx=0x7f7de000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0018070 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de00072d0 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de00047e0 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df4155610 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.391+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df4155bd0 con 0x7f7df40cfa40 2024-03-04T12:12:36.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.392+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df4074f60 con 0x7f7df40cfa40 2024-03-04T12:12:36.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.392+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7de001b070 con 0x7f7df40cfa40 2024-03-04T12:12:36.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.393+0000 7f7df17fa640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 0x7f7dd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.393+0000 7f7df8cf5640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 0x7f7dd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.393+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7de0089a60 con 0x7f7df40cfa40 2024-03-04T12:12:36.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.393+0000 7f7df8cf5640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 0x7f7dd806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7de8034000 tx=0x7f7de8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.397+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7de0058570 con 0x7f7df40cfa40 2024-03-04T12:12:36.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.569+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7df40b1ab0 con 0x7f7df40cfa40 2024-03-04T12:12:36.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.570+0000 7f7df17fa640 1 -- 172.21.15.148:0/757335816 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7de0057e70 con 0x7f7df40cfa40 2024-03-04T12:12:36.571 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:36.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 msgr2=0x7f7dd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 0x7f7dd806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7de8034000 tx=0x7f7de8008030 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 msgr2=0x7f7df40c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c800 tx=0x7f7de000ccc0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 shutdown_connections 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7dd806b4d0 0x7f7dd806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df415df00 0x7f7df41563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df40cfa40 0x7f7df40c5bf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 --2- 172.21.15.148:0/757335816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df406fa80 0x7f7df414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 >> 172.21.15.148:0/757335816 conn(0x7f7df40b2250 msgr2=0x7f7df40d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 shutdown_connections 2024-03-04T12:12:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.572+0000 7f7dfaf59640 1 -- 172.21.15.148:0/757335816 wait complete. 2024-03-04T12:12:36.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:36.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.683+0000 7fbb2c0de640 1 Processor -- start 2024-03-04T12:12:36.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 -- start start 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb240ce0a0 0x7fbb241618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb24161de0 0x7fbb240cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb24162200 con 0x7fbb2415dcd0 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240cc0c0 con 0x7fbb240ce0a0 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb2c0de640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240cf200 con 0x7fbb24161de0 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb29679640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb240ce0a0 0x7fbb241618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb29e7a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb2415dcd0 0x7fbb240cfeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38264/0 (socket says 172.21.15.148:38264) 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.684+0000 7fbb29e7a640 1 -- 172.21.15.148:0/852242232 learned_addr learned my addr 172.21.15.148:0/852242232 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb2a67b640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb24161de0 0x7fbb240cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb29679640 1 -- 172.21.15.148:0/852242232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb24161de0 msgr2=0x7fbb240cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb29679640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb24161de0 0x7fbb240cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb29679640 1 -- 172.21.15.148:0/852242232 >> v1:172.21.15.148:6789/0 conn(0x7fbb2415dcd0 legacy=0x7fbb240cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb29679640 1 -- 172.21.15.148:0/852242232 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240ce680 con 0x7fbb240ce0a0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.685+0000 7fbb28e78640 1 -- 172.21.15.148:0/852242232 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126337623 0 0) 0x7fbb24162200 con 0x7fbb2415dcd0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb29679640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240ce0a0 0x7fbb241618a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb200097a0 tx=0x7fbb2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc616e4eef9fb1f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb28e78640 1 -- 172.21.15.148:0/852242232 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20035020 con 0x7fbb240ce0a0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb28e78640 1 -- 172.21.15.148:0/852242232 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20031830 con 0x7fbb240ce0a0 2024-03-04T12:12:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb28e78640 1 -- 172.21.15.148:0/852242232 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031c00 con 0x7fbb240ce0a0 2024-03-04T12:12:36.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 -- 172.21.15.148:0/852242232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240ce0a0 msgr2=0x7fbb241618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240ce0a0 0x7fbb241618a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb200097a0 tx=0x7fbb2002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 -- 172.21.15.148:0/852242232 shutdown_connections 2024-03-04T12:12:36.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb24161de0 0x7fbb240cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/852242232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240ce0a0 0x7fbb241618a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 -- 172.21.15.148:0/852242232 >> 172.21.15.148:0/852242232 conn(0x7fbb240b3480 msgr2=0x7fbb240b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 -- 172.21.15.148:0/852242232 shutdown_connections 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.686+0000 7fbb2c0de640 1 -- 172.21.15.148:0/852242232 wait complete. 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.687+0000 7fbb2c0de640 1 Processor -- start 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 -- start start 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb240ce0a0 0x7fbb240b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb2415dcd0 0x7fbb240c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240cef20 con 0x7fbb24161de0 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240cec40 con 0x7fbb240ce0a0 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2c0de640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2406bff0 con 0x7fbb2415dcd0 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2a67b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2a67b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59064/0 (socket says 172.21.15.148:59064) 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.688+0000 7fbb2a67b640 1 -- 172.21.15.148:0/1021568862 learned_addr learned my addr 172.21.15.148:0/1021568862 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 -- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb2415dcd0 msgr2=0x7fbb240c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb29679640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb2415dcd0 0x7fbb240c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb29e7a640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb240ce0a0 0x7fbb240b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb2415dcd0 0x7fbb240c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 -- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb240ce0a0 msgr2=0x7fbb240b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb240ce0a0 0x7fbb240b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 -- 172.21.15.148:0/1021568862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240c8a00 con 0x7fbb24161de0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2a67b640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007e70 tx=0x7fbb1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c014620 con 0x7fbb24161de0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c019070 con 0x7fbb24161de0 2024-03-04T12:12:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb240c8ce0 con 0x7fbb24161de0 2024-03-04T12:12:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c014a20 con 0x7fbb24161de0 2024-03-04T12:12:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.689+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb240c92a0 con 0x7fbb24161de0 2024-03-04T12:12:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.690+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb24074f60 con 0x7fbb24161de0 2024-03-04T12:12:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.690+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb1c014bc0 con 0x7fbb24161de0 2024-03-04T12:12:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.691+0000 7fbb167fc640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 0x7fbb0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.691+0000 7fbb29e7a640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 0x7fbb0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.691+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbb1c08af20 con 0x7fbb24161de0 2024-03-04T12:12:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.691+0000 7fbb29e7a640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 0x7fbb0806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.695+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb1c0599a0 con 0x7fbb24161de0 2024-03-04T12:12:36.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.867+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb24152360 con 0x7fbb24161de0 2024-03-04T12:12:36.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.868+0000 7fbb167fc640 1 -- 172.21.15.148:0/1021568862 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fbb1c0592a0 con 0x7fbb24161de0 2024-03-04T12:12:36.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:36.870 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 msgr2=0x7fbb0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 0x7fbb0806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 msgr2=0x7fbb241f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007e70 tx=0x7fbb1c00e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 shutdown_connections 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbb0806b4d0 0x7fbb0806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24161de0 0x7fbb241f71a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb2415dcd0 0x7fbb240c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 --2- 172.21.15.148:0/1021568862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb240ce0a0 0x7fbb240b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 >> 172.21.15.148:0/1021568862 conn(0x7fbb240b3480 msgr2=0x7fbb240cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 shutdown_connections 2024-03-04T12:12:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.870+0000 7fbb2c0de640 1 -- 172.21.15.148:0/1021568862 wait complete. 2024-03-04T12:12:36.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:36.883 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:36.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 Processor -- start 2024-03-04T12:12:36.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 -- start start 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49280cfc70 0x7f49280cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 0x7f49280b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280cebd0 con 0x7f49281450e0 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280cc7b0 con 0x7f49280cfc70 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.985+0000 7f492dc9a640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280cf7e0 con 0x7f49280cc340 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49280cfc70 0x7f49280cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49280cfc70 0x7f49280cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59080/0 (socket says 172.21.15.148:59080) 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 -- 172.21.15.148:0/1464298642 learned_addr learned my addr 172.21.15.148:0/1464298642 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4927fff640 1 --2- 172.21.15.148:0/1464298642 >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 0x7f49280b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 -- 172.21.15.148:0/1464298642 >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 msgr2=0x7f49280b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 --2- 172.21.15.148:0/1464298642 >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 0x7f49280b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 -- 172.21.15.148:0/1464298642 >> v1:172.21.15.191:6789/0 conn(0x7f49281450e0 legacy=0x7f492815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4926ffd640 1 -- 172.21.15.148:0/1464298642 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280ceeb0 con 0x7f49280cfc70 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.986+0000 7f4927fff640 1 --2- 172.21.15.148:0/1464298642 >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 0x7f49280b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f4926ffd640 1 --2- 172.21.15.148:0/1464298642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280cfc70 0x7f49280cbe00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49280aebd0 tx=0x7f491c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d0061909445032 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f49267fc640 1 -- 172.21.15.148:0/1464298642 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c034070 con 0x7f49280cfc70 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f49267fc640 1 -- 172.21.15.148:0/1464298642 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c030860 con 0x7f49280cfc70 2024-03-04T12:12:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f49267fc640 1 -- 172.21.15.148:0/1464298642 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c030c30 con 0x7f49280cfc70 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 -- 172.21.15.148:0/1464298642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280cfc70 msgr2=0x7f49280cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1464298642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280cfc70 0x7f49280cbe00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49280aebd0 tx=0x7f491c02b9a0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 -- 172.21.15.148:0/1464298642 shutdown_connections 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1464298642 >> v2:172.21.15.191:3300/0 conn(0x7f49280cc340 0x7f49280b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1464298642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280cfc70 0x7f49280cbe00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 -- 172.21.15.148:0/1464298642 >> 172.21.15.148:0/1464298642 conn(0x7f492806acd0 msgr2=0x7f492806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 -- 172.21.15.148:0/1464298642 shutdown_connections 2024-03-04T12:12:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.987+0000 7f492dc9a640 1 -- 172.21.15.148:0/1464298642 wait complete. 2024-03-04T12:12:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.988+0000 7f492dc9a640 1 Processor -- start 2024-03-04T12:12:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 -- start start 2024-03-04T12:12:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 0x7f49281409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 0x7f492813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49281454c0 con 0x7f49281450e0 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280cf470 con 0x7f49280cfc70 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f492dc9a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49280ccd10 con 0x7f49280cc340 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.989+0000 7f4927fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59096/0 (socket says 172.21.15.148:59096) 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 -- 172.21.15.148:0/1432262020 learned_addr learned my addr 172.21.15.148:0/1432262020 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f49277fe640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 0x7f49281409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 -- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 msgr2=0x7f49281409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 0x7f49281409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 -- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 msgr2=0x7f492813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4926ffd640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 0x7f492813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 0x7f492813ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 -- 172.21.15.148:0/1432262020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928140f10 con 0x7f49281450e0 2024-03-04T12:12:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f49277fe640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 0x7f49281409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4926ffd640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 0x7f492813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f4927fff640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4920007df0 tx=0x7f492000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492001a070 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920003990 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920003d80 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492814c270 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.990+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492814c830 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.991+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4928074f60 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.992+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f492001d070 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.992+0000 7f490ffff640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.992+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f492008a7f0 con 0x7f49281450e0 2024-03-04T12:12:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.992+0000 7f49277fe640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.993+0000 7f49277fe640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 0x7f491406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4918001680 tx=0x7f4918009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:36.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:36.996+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4928140f10 con 0x7f49281450e0 2024-03-04T12:12:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.167+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49280cda60 con 0x7f49281450e0 2024-03-04T12:12:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.168+0000 7f490ffff640 1 -- 172.21.15.148:0/1432262020 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4920010070 con 0x7f49281450e0 2024-03-04T12:12:37.169 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 msgr2=0x7f491406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 0x7f491406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4918001680 tx=0x7f4918009040 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 msgr2=0x7f49281e4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4920007df0 tx=0x7f492000d4a0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 shutdown_connections 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49281450e0 0x7f49281e4d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49280cfc70 0x7f492813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 --2- 172.21.15.148:0/1432262020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49280cc340 0x7f49281409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 >> 172.21.15.148:0/1432262020 conn(0x7f492806acd0 msgr2=0x7f49280b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 shutdown_connections 2024-03-04T12:12:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.170+0000 7f492dc9a640 1 -- 172.21.15.148:0/1432262020 wait complete. 2024-03-04T12:12:37.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.284+0000 7fda05529640 1 Processor -- start 2024-03-04T12:12:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.284+0000 7fda05529640 1 -- start start 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fda05529640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda000ccbd0 0x7fda000b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fda05529640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 0x7fda000b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fda05529640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0015ca50 con 0x7fda0015c2e0 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fda05529640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0015d2f0 con 0x7fda000ccbd0 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fda05529640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0015d6d0 con 0x7fda000ceec0 2024-03-04T12:12:37.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fd9ff7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 0x7fda000b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fd9feffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda000ccbd0 0x7fda000b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fd9fe7fc640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fda0015c2e0 0x7fda000ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:38464/0 (socket says 172.21.15.148:38464) 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.285+0000 7fd9fe7fc640 1 -- 172.21.15.148:0/1114440108 learned_addr learned my addr 172.21.15.148:0/1114440108 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.286+0000 7fd9ff7fe640 1 -- 172.21.15.148:0/1114440108 >> v2:172.21.15.191:3300/0 conn(0x7fda000ccbd0 msgr2=0x7fda000b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.286+0000 7fd9ff7fe640 1 --2- 172.21.15.148:0/1114440108 >> v2:172.21.15.191:3300/0 conn(0x7fda000ccbd0 0x7fda000b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.286+0000 7fd9ff7fe640 1 -- 172.21.15.148:0/1114440108 >> v1:172.21.15.191:6789/0 conn(0x7fda0015c2e0 legacy=0x7fda000ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.286+0000 7fd9ff7fe640 1 -- 172.21.15.148:0/1114440108 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0015cd30 con 0x7fda000ceec0 2024-03-04T12:12:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fd9ff7fe640 1 --2- 172.21.15.148:0/1114440108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 0x7fda000b0560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf276fe10fb10d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fd9fdffb640 1 -- 172.21.15.148:0/1114440108 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8035020 con 0x7fda000ceec0 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fd9fdffb640 1 -- 172.21.15.148:0/1114440108 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031830 con 0x7fda000ceec0 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fd9fdffb640 1 -- 172.21.15.148:0/1114440108 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031c00 con 0x7fda000ceec0 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 -- 172.21.15.148:0/1114440108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 msgr2=0x7fda000b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 --2- 172.21.15.148:0/1114440108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 0x7fda000b0560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 -- 172.21.15.148:0/1114440108 shutdown_connections 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 --2- 172.21.15.148:0/1114440108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda000ceec0 0x7fda000b0560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 --2- 172.21.15.148:0/1114440108 >> v2:172.21.15.191:3300/0 conn(0x7fda000ccbd0 0x7fda000b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 -- 172.21.15.148:0/1114440108 >> 172.21.15.148:0/1114440108 conn(0x7fda00066dc0 msgr2=0x7fda000671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 -- 172.21.15.148:0/1114440108 shutdown_connections 2024-03-04T12:12:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.287+0000 7fda05529640 1 -- 172.21.15.148:0/1114440108 wait complete. 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 Processor -- start 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 -- start start 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda000ccbd0 0x7fda001548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda0015c2e0 0x7fda000c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000ccfb0 con 0x7fda000ceec0 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0015c760 con 0x7fda000ccbd0 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fda05529640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000b0aa0 con 0x7fda0015c2e0 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fd9fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.289+0000 7fd9fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59110/0 (socket says 172.21.15.148:59110) 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 -- 172.21.15.148:0/775664618 learned_addr learned my addr 172.21.15.148:0/775664618 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9ff7fe640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda0015c2e0 0x7fda000c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9feffd640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda000ccbd0 0x7fda001548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 -- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda0015c2e0 msgr2=0x7fda000c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda0015c2e0 0x7fda000c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 -- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda000ccbd0 msgr2=0x7fda001548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda000ccbd0 0x7fda001548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 -- 172.21.15.148:0/775664618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00154de0 con 0x7fda000ceec0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9fe7fc640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c3c0 tx=0x7fd9f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4017070 con 0x7fda000ceec0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4005ce0 con 0x7fda000ceec0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda001568b0 con 0x7fda000ceec0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40072f0 con 0x7fda000ceec0 2024-03-04T12:12:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.290+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda00156e70 con 0x7fda000ceec0 2024-03-04T12:12:37.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.291+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00074f60 con 0x7fda000ceec0 2024-03-04T12:12:37.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.292+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd9f401a070 con 0x7fda000ceec0 2024-03-04T12:12:37.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.292+0000 7fd9e77fe640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.292+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd9f4089930 con 0x7fda000ceec0 2024-03-04T12:12:37.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.292+0000 7fd9feffd640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.292+0000 7fd9feffd640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00113e0 tx=0x7fd9f000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.296+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9f4057d60 con 0x7fda000ceec0 2024-03-04T12:12:37.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.468+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda000b0880 con 0x7fda000ceec0 2024-03-04T12:12:37.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.469+0000 7fd9e77fe640 1 -- 172.21.15.148:0/775664618 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9f405be00 con 0x7fda000ceec0 2024-03-04T12:12:37.471 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 msgr2=0x7fd9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00113e0 tx=0x7fd9f000b040 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 msgr2=0x7fda000c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c3c0 tx=0x7fd9f400c880 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 shutdown_connections 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda0015c2e0 0x7fda000c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ceec0 0x7fda000c7bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 --2- 172.21.15.148:0/775664618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda000ccbd0 0x7fda001548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 >> 172.21.15.148:0/775664618 conn(0x7fda00066dc0 msgr2=0x7fda000b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 shutdown_connections 2024-03-04T12:12:37.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.471+0000 7fda05529640 1 -- 172.21.15.148:0/775664618 wait complete. 2024-03-04T12:12:37.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 Processor -- start 2024-03-04T12:12:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 -- start start 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8f740d34a0 0x7f8f74161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f740d2400 con 0x7f8f740b5700 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740d26e0 con 0x7f8f740d34a0 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.586+0000 7f8f7a71b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740d29c0 con 0x7f8f740b9410 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:52312/0 (socket says 172.21.15.148:52312) 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2942133066 learned_addr learned my addr 172.21.15.148:0/2942133066 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f737fe640 1 --2- 172.21.15.148:0/2942133066 >> v2:172.21.15.191:3300/0 conn(0x7f8f740d34a0 0x7f8f74161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f72ffd640 1 -- 172.21.15.148:0/2942133066 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339158642 0 0) 0x7f8f740d2400 con 0x7f8f740b5700 2024-03-04T12:12:37.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2942133066 >> v2:172.21.15.191:3300/0 conn(0x7f8f740d34a0 msgr2=0x7f8f74161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:37.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 --2- 172.21.15.148:0/2942133066 >> v2:172.21.15.191:3300/0 conn(0x7f8f740d34a0 0x7f8f74161be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2942133066 >> v1:172.21.15.191:6789/0 conn(0x7f8f740b5700 legacy=0x7f8f740d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.587+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2942133066 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740d2ca0 con 0x7f8f740b9410 2024-03-04T12:12:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.588+0000 7f8f78cb8640 1 --2- 172.21.15.148:0/2942133066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009e50 tx=0x7f8f6c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284862552f3e9830 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.588+0000 7f8f72ffd640 1 -- 172.21.15.148:0/2942133066 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c02ca70 con 0x7f8f740b9410 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.588+0000 7f8f72ffd640 1 -- 172.21.15.148:0/2942133066 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c02cc10 con 0x7f8f740b9410 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.588+0000 7f8f72ffd640 1 -- 172.21.15.148:0/2942133066 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0343f0 con 0x7f8f740b9410 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2942133066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 msgr2=0x7f8f74162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2942133066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009e50 tx=0x7f8f6c02bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2942133066 shutdown_connections 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2942133066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b9410 0x7f8f74162120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2942133066 >> v2:172.21.15.191:3300/0 conn(0x7f8f740d34a0 0x7f8f74161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2942133066 >> 172.21.15.148:0/2942133066 conn(0x7f8f740d0490 msgr2=0x7f8f740d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2942133066 shutdown_connections 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.589+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2942133066 wait complete. 2024-03-04T12:12:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.590+0000 7f8f7a71b640 1 Processor -- start 2024-03-04T12:12:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.590+0000 7f8f7a71b640 1 -- start start 2024-03-04T12:12:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b5700 0x7f8f740b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f740b9410 0x7f8f7415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74162660 con 0x7f8f740d34a0 2024-03-04T12:12:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740b5cf0 con 0x7f8f740b9410 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f7a71b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f741627e0 con 0x7f8f740b5700 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59118/0 (socket says 172.21.15.148:59118) 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2070029104 learned_addr learned my addr 172.21.15.148:0/2070029104 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f737fe640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f740b9410 0x7f8f7415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f73fff640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b5700 0x7f8f740b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b5700 msgr2=0x7f8f740b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b5700 0x7f8f740b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f740b9410 msgr2=0x7f8f7415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f740b9410 0x7f8f7415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 -- 172.21.15.148:0/2070029104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7415c210 con 0x7f8f740d34a0 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.591+0000 7f8f78cb8640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c036040 tx=0x7f8f6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.592+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c043030 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.592+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c03e450 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.592+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009510 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.592+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740cb590 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.592+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f740cb830 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.593+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74074f60 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.593+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8f6c046070 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.593+0000 7f8f70ff9640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.593+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f6c0ac7f0 con 0x7f8f740d34a0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.593+0000 7f8f73fff640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.594+0000 7f8f73fff640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 0x7f8f6006d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000f60 tx=0x7f8f64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.598+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f6c04d070 con 0x7f8f740d34a0 2024-03-04T12:12:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.771+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f741f7790 con 0x7f8f740d34a0 2024-03-04T12:12:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.771+0000 7f8f70ff9640 1 -- 172.21.15.148:0/2070029104 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f7415c210 con 0x7f8f740d34a0 2024-03-04T12:12:37.773 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:37.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 msgr2=0x7f8f6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 0x7f8f6006d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000f60 tx=0x7f8f64009040 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 msgr2=0x7f8f741f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c036040 tx=0x7f8f6c004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 shutdown_connections 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740d34a0 0x7f8f741f68e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f740b9410 0x7f8f7415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 --2- 172.21.15.148:0/2070029104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f740b5700 0x7f8f740b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 >> 172.21.15.148:0/2070029104 conn(0x7f8f740d0490 msgr2=0x7f8f740b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 shutdown_connections 2024-03-04T12:12:37.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.774+0000 7f8f7a71b640 1 -- 172.21.15.148:0/2070029104 wait complete. 2024-03-04T12:12:37.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:37.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.888+0000 7f92baf44640 1 Processor -- start 2024-03-04T12:12:37.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 -- start start 2024-03-04T12:12:37.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92b4071320 0x7f92b4071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f92b406d810 0x7f92b4150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b4071c40 con 0x7f92b406d3c0 2024-03-04T12:12:37.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b406dd10 con 0x7f92b4071320 2024-03-04T12:12:37.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92baf44640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b406dff0 con 0x7f92b406d810 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92b8ce0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92b4071320 0x7f92b4071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92b8ce0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92b4071320 0x7f92b4071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59126/0 (socket says 172.21.15.148:59126) 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.889+0000 7f92b8ce0640 1 -- 172.21.15.148:0/1089838084 learned_addr learned my addr 172.21.15.148:0/1089838084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.890+0000 7f92b94e1640 1 --2- 172.21.15.148:0/1089838084 >> v2:172.21.15.191:3300/0 conn(0x7f92b406d810 0x7f92b4150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.890+0000 7f92b8ce0640 1 -- 172.21.15.148:0/1089838084 >> v2:172.21.15.191:3300/0 conn(0x7f92b406d810 msgr2=0x7f92b4150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.890+0000 7f92b8ce0640 1 --2- 172.21.15.148:0/1089838084 >> v2:172.21.15.191:3300/0 conn(0x7f92b406d810 0x7f92b4150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.890+0000 7f92b8ce0640 1 -- 172.21.15.148:0/1089838084 >> v1:172.21.15.191:6789/0 conn(0x7f92b406d3c0 legacy=0x7f92b41508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.890+0000 7f92b8ce0640 1 -- 172.21.15.148:0/1089838084 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b406e2d0 con 0x7f92b4071320 2024-03-04T12:12:37.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92b8ce0640 1 --2- 172.21.15.148:0/1089838084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b4071320 0x7f92b4071700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92b4158060 tx=0x7f92a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963a379c374b8e61 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92b37fe640 1 -- 172.21.15.148:0/1089838084 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a402e070 con 0x7f92b4071320 2024-03-04T12:12:37.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92b37fe640 1 -- 172.21.15.148:0/1089838084 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4037070 con 0x7f92b4071320 2024-03-04T12:12:37.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92b37fe640 1 -- 172.21.15.148:0/1089838084 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4031a00 con 0x7f92b4071320 2024-03-04T12:12:37.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 -- 172.21.15.148:0/1089838084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b4071320 msgr2=0x7f92b4071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 --2- 172.21.15.148:0/1089838084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b4071320 0x7f92b4071700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92b4158060 tx=0x7f92a402b880 comp rx=0 tx=0).stop 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 -- 172.21.15.148:0/1089838084 shutdown_connections 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 --2- 172.21.15.148:0/1089838084 >> v2:172.21.15.191:3300/0 conn(0x7f92b406d810 0x7f92b4150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 --2- 172.21.15.148:0/1089838084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b4071320 0x7f92b4071700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 -- 172.21.15.148:0/1089838084 >> 172.21.15.148:0/1089838084 conn(0x7f92b406acd0 msgr2=0x7f92b406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 -- 172.21.15.148:0/1089838084 shutdown_connections 2024-03-04T12:12:37.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.891+0000 7f92baf44640 1 -- 172.21.15.148:0/1089838084 wait complete. 2024-03-04T12:12:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.892+0000 7f92baf44640 1 Processor -- start 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.892+0000 7f92baf44640 1 -- start start 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92b406d3c0 0x7f92b41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b4071320 0x7f92b406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40cf7e0 con 0x7f92b406d810 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40cf660 con 0x7f92b4071320 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92baf44640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40cf960 con 0x7f92b406d3c0 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59136/0 (socket says 172.21.15.148:59136) 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.893+0000 7f92b3fff640 1 -- 172.21.15.148:0/193331616 learned_addr learned my addr 172.21.15.148:0/193331616 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 -- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92b406d3c0 msgr2=0x7f92b41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92b406d3c0 0x7f92b41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 -- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b4071320 msgr2=0x7f92b406ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b4071320 0x7f92b406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 -- 172.21.15.148:0/193331616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40cefb0 con 0x7f92b406d810 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b3fff640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b3f0 tx=0x7f92a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8015020 con 0x7f92b406d810 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a8011830 con 0x7f92b406d810 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41f3fb0 con 0x7f92b406d810 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8011c20 con 0x7f92b406d810 2024-03-04T12:12:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.894+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b41f3ca0 con 0x7f92b406d810 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.895+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b41f43a0 con 0x7f92b406d810 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.896+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f92a8018020 con 0x7f92b406d810 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.896+0000 7f92b17fa640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 0x7f92a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.896+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f92a808a500 con 0x7f92b406d810 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.896+0000 7f92b8ce0640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 0x7f92a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.896+0000 7f92b8ce0640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 0x7f92a006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036000 tx=0x7f92a4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:37.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:37.900+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92a801f020 con 0x7f92b406d810 2024-03-04T12:12:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.071+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f92b4074f60 con 0x7f92b406d810 2024-03-04T12:12:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.072+0000 7f92b17fa640 1 -- 172.21.15.148:0/193331616 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f92b40cefb0 con 0x7f92b406d810 2024-03-04T12:12:38.073 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.073+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 msgr2=0x7f92a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.073+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 0x7f92a006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036000 tx=0x7f92a4030000 comp rx=0 tx=0).stop 2024-03-04T12:12:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.073+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 msgr2=0x7f92b40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.073+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b3f0 tx=0x7f92a800b8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 shutdown_connections 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f92a006b4d0 0x7f92a006d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b4071320 0x7f92b406ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92b406d810 0x7f92b40cea70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 --2- 172.21.15.148:0/193331616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92b406d3c0 0x7f92b41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 >> 172.21.15.148:0/193331616 conn(0x7f92b406acd0 msgr2=0x7f92b40b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 shutdown_connections 2024-03-04T12:12:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.074+0000 7f92baf44640 1 -- 172.21.15.148:0/193331616 wait complete. 2024-03-04T12:12:38.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.185+0000 7fdfbfb7d640 1 Processor -- start 2024-03-04T12:12:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.185+0000 7fdfbfb7d640 1 -- start start 2024-03-04T12:12:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbfb7d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb815b960 0x7fdfb80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbfb7d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfb80b0e10 0x7fdfb80b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbfb7d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb80ccad0 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbfb7d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb80cf860 con 0x7fdfb80b0e10 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbfb7d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80d0100 con 0x7fdfb815b960 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbe11a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfb80b0e10 0x7fdfb80b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbe11a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfb80b0e10 0x7fdfb80b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59142/0 (socket says 172.21.15.148:59142) 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.186+0000 7fdfbe11a640 1 -- 172.21.15.148:0/3405153910 learned_addr learned my addr 172.21.15.148:0/3405153910 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbd919640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb815b960 0x7fdfb80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbe11a640 1 -- 172.21.15.148:0/3405153910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb815b960 msgr2=0x7fdfb80d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbe11a640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb815b960 0x7fdfb80d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbe11a640 1 -- 172.21.15.148:0/3405153910 >> v1:172.21.15.191:6789/0 conn(0x7fdfb80d0ad0 legacy=0x7fdfb80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbe11a640 1 -- 172.21.15.148:0/3405153910 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80cfab0 con 0x7fdfb80b0e10 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.187+0000 7fdfbe11a640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80b0e10 0x7fdfb80b47c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009870 tx=0x7fdfa802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407024d583452ee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbc917640 1 -- 172.21.15.148:0/3405153910 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80317c0 con 0x7fdfb80b0e10 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbc917640 1 -- 172.21.15.148:0/3405153910 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8037070 con 0x7fdfb80b0e10 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbc917640 1 -- 172.21.15.148:0/3405153910 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8031b80 con 0x7fdfb80b0e10 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/3405153910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80b0e10 msgr2=0x7fdfb80b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80b0e10 0x7fdfb80b47c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009870 tx=0x7fdfa802b860 comp rx=0 tx=0).stop 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/3405153910 shutdown_connections 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80b0e10 0x7fdfb80b47c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/3405153910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb815b960 0x7fdfb80d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/3405153910 >> 172.21.15.148:0/3405153910 conn(0x7fdfb806f5f0 msgr2=0x7fdfb806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/3405153910 shutdown_connections 2024-03-04T12:12:38.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.188+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/3405153910 wait complete. 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.189+0000 7fdfbfb7d640 1 Processor -- start 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.189+0000 7fdfbfb7d640 1 -- start start 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb80b0e10 0x7fdfb80aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfb815b960 0x7fdfb80a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815bec0 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815bd40 con 0x7fdfb815b960 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbfb7d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b4d00 con 0x7fdfb80b0e10 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59146/0 (socket says 172.21.15.148:59146) 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 -- 172.21.15.148:0/4198646951 learned_addr learned my addr 172.21.15.148:0/4198646951 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd919640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb80b0e10 0x7fdfb80aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 -- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb80b0e10 msgr2=0x7fdfb80aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbe11a640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfb815b960 0x7fdfb80a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb80b0e10 0x7fdfb80aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 -- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfb815b960 msgr2=0x7fdfb80a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfb815b960 0x7fdfb80a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.190+0000 7fdfbd118640 1 -- 172.21.15.148:0/4198646951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb81f2190 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfbd118640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c7c0 tx=0x7fdfac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac016020 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac00aa50 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb80a8470 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac012420 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.191+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb80a8a30 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.192+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8074f60 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.192+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdfac019020 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.193+0000 7fdfb67fc640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.193+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdfac08a5e0 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.193+0000 7fdfbd919640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.193+0000 7fdfbd919640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 0x7fdfa406d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdfb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.197+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfac020030 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.367+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfb806e620 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.368+0000 7fdfb67fc640 1 -- 172.21.15.148:0/4198646951 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fdfac0590f0 con 0x7fdfb80d0ad0 2024-03-04T12:12:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:38.370 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 msgr2=0x7fdfa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 0x7fdfa406d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdfb0008040 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 msgr2=0x7fdfb815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c7c0 tx=0x7fdfac00cc80 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 shutdown_connections 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfb815b960 0x7fdfb80a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80d0ad0 0x7fdfb815c040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 --2- 172.21.15.148:0/4198646951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfb80b0e10 0x7fdfb80aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 >> 172.21.15.148:0/4198646951 conn(0x7fdfb806f5f0 msgr2=0x7fdfb80b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 shutdown_connections 2024-03-04T12:12:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.370+0000 7fdfbfb7d640 1 -- 172.21.15.148:0/4198646951 wait complete. 2024-03-04T12:12:38.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:38.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.486+0000 7fa4cddf1640 1 Processor -- start 2024-03-04T12:12:38.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 -- start start 2024-03-04T12:12:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa4c80b70d0 0x7fa4c80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4c80d1770 0x7fa4c80d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80d1520 con 0x7fa4c815bbf0 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c80d2110 con 0x7fa4c80d1770 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4cddf1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c80cfa30 con 0x7fa4c80b70d0 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4c7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4c80d1770 0x7fa4c80d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4c7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4c80d1770 0x7fa4c80d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59152/0 (socket says 172.21.15.148:59152) 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.487+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2082051843 learned_addr learned my addr 172.21.15.148:0/2082051843 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.488+0000 7fa4beffd640 1 --2- 172.21.15.148:0/2082051843 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80b70d0 0x7fa4c80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.488+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2082051843 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80b70d0 msgr2=0x7fa4c80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.488+0000 7fa4c7fff640 1 --2- 172.21.15.148:0/2082051843 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80b70d0 0x7fa4c80cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.488+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2082051843 >> v1:172.21.15.191:6789/0 conn(0x7fa4c815bbf0 legacy=0x7fa4c80d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:38.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.488+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2082051843 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80cfd10 con 0x7fa4c80d1770 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4c7fff640 1 --2- 172.21.15.148:0/2082051843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c80d1770 0x7fa4c80d1bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8072e70 tx=0x7fa4c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9285eec6e3e93c5d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4c6ffd640 1 -- 172.21.15.148:0/2082051843 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c002c9f0 con 0x7fa4c80d1770 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4c6ffd640 1 -- 172.21.15.148:0/2082051843 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c002cb90 con 0x7fa4c80d1770 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4c6ffd640 1 -- 172.21.15.148:0/2082051843 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00343f0 con 0x7fa4c80d1770 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2082051843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c80d1770 msgr2=0x7fa4c80d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2082051843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c80d1770 0x7fa4c80d1bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8072e70 tx=0x7fa4c002be50 comp rx=0 tx=0).stop 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2082051843 shutdown_connections 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2082051843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c80d1770 0x7fa4c80d1bd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2082051843 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80b70d0 0x7fa4c80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2082051843 >> 172.21.15.148:0/2082051843 conn(0x7fa4c80b3330 msgr2=0x7fa4c80b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2082051843 shutdown_connections 2024-03-04T12:12:38.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.489+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2082051843 wait complete. 2024-03-04T12:12:38.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.490+0000 7fa4cddf1640 1 Processor -- start 2024-03-04T12:12:38.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 -- start start 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 0x7fa4c80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 0x7fa4c81f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80d05b0 con 0x7fa4c815bbf0 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80d02d0 con 0x7fa4c80b70d0 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4cddf1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c815bfd0 con 0x7fa4c80d1770 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59154/0 (socket says 172.21.15.148:59154) 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.491+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2411937260 learned_addr learned my addr 172.21.15.148:0/2411937260 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4beffd640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 0x7fa4c81f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c77fe640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 0x7fa4c80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 msgr2=0x7fa4c81f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 0x7fa4c81f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 msgr2=0x7fa4c80c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 0x7fa4c80c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 -- 172.21.15.148:0/2411937260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c81f3ed0 con 0x7fa4c815bbf0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c77fe640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 0x7fa4c80c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4beffd640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 0x7fa4c81f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c7fff640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009f40 tx=0x7fa4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0043020 con 0x7fa4c815bbf0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c003e450 con 0x7fa4c815bbf0 2024-03-04T12:12:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c81f41b0 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c003e820 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.492+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c80c8d10 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.493+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c8074f60 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.493+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa4c0046020 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.494+0000 7fa4c4ff9640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.494+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa4c00ad1c0 con 0x7fa4c815bbf0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.494+0000 7fa4c77fe640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.494+0000 7fa4c77fe640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 0x7fa4a806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b67a0 tx=0x7fa4b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.498+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4c004d020 con 0x7fa4c815bbf0 2024-03-04T12:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.670+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4c81f4380 con 0x7fa4c815bbf0 2024-03-04T12:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.671+0000 7fa4c4ff9640 1 -- 172.21.15.148:0/2411937260 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4c007bcd0 con 0x7fa4c815bbf0 2024-03-04T12:12:38.672 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 msgr2=0x7fa4a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 0x7fa4a806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b67a0 tx=0x7fa4b8009040 comp rx=0 tx=0).stop 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 msgr2=0x7fa4c806bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009f40 tx=0x7fa4c0004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 shutdown_connections 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4c815bbf0 0x7fa4c806bde0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c80d1770 0x7fa4c81f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 --2- 172.21.15.148:0/2411937260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c80b70d0 0x7fa4c80c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 >> 172.21.15.148:0/2411937260 conn(0x7fa4c80b3330 msgr2=0x7fa4c80b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 shutdown_connections 2024-03-04T12:12:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.673+0000 7fa4cddf1640 1 -- 172.21.15.148:0/2411937260 wait complete. 2024-03-04T12:12:38.686 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.788+0000 7f3430659640 1 Processor -- start 2024-03-04T12:12:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.788+0000 7f3430659640 1 -- start start 2024-03-04T12:12:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f3430659640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f342806bce0 0x7f34280728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f3430659640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f342815bba0 0x7f34280700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f3430659640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342815cec0 con 0x7f342806c0c0 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f3430659640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342815c100 con 0x7f342815bba0 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f3430659640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342815c3e0 con 0x7f342806bce0 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342dbf4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f342815bba0 0x7f34280700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342ebf6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f342806c0c0 0x7f342815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38276/0 (socket says 172.21.15.148:38276) 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342ebf6640 1 -- 172.21.15.148:0/4088010029 learned_addr learned my addr 172.21.15.148:0/4088010029 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342e3f5640 1 --2- 172.21.15.148:0/4088010029 >> v2:172.21.15.191:3300/0 conn(0x7f342806bce0 0x7f34280728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342dbf4640 1 -- 172.21.15.148:0/4088010029 >> v2:172.21.15.191:3300/0 conn(0x7f342806bce0 msgr2=0x7f34280728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342dbf4640 1 --2- 172.21.15.148:0/4088010029 >> v2:172.21.15.191:3300/0 conn(0x7f342806bce0 0x7f34280728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342dbf4640 1 -- 172.21.15.148:0/4088010029 >> v1:172.21.15.148:6789/0 conn(0x7f342806c0c0 legacy=0x7f342815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342d3f3640 1 -- 172.21.15.148:0/4088010029 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422787807 0 0) 0x7f342815cec0 con 0x7f342806c0c0 2024-03-04T12:12:38.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.789+0000 7f342dbf4640 1 -- 172.21.15.148:0/4088010029 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342815c6c0 con 0x7f342815bba0 2024-03-04T12:12:38.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.790+0000 7f342dbf4640 1 --2- 172.21.15.148:0/4088010029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342815bba0 0x7f34280700a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f342806c870 tx=0x7f341802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c8fe243026df8c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.790+0000 7f342d3f3640 1 -- 172.21.15.148:0/4088010029 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180315a0 con 0x7f342815bba0 2024-03-04T12:12:38.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.790+0000 7f342d3f3640 1 -- 172.21.15.148:0/4088010029 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418037070 con 0x7f342815bba0 2024-03-04T12:12:38.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.790+0000 7f342d3f3640 1 -- 172.21.15.148:0/4088010029 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418031960 con 0x7f342815bba0 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 -- 172.21.15.148:0/4088010029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342815bba0 msgr2=0x7f34280700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 --2- 172.21.15.148:0/4088010029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342815bba0 0x7f34280700a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f342806c870 tx=0x7f341802b840 comp rx=0 tx=0).stop 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 -- 172.21.15.148:0/4088010029 shutdown_connections 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 --2- 172.21.15.148:0/4088010029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342815bba0 0x7f34280700a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 --2- 172.21.15.148:0/4088010029 >> v2:172.21.15.191:3300/0 conn(0x7f342806bce0 0x7f34280728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 -- 172.21.15.148:0/4088010029 >> 172.21.15.148:0/4088010029 conn(0x7f3428071bc0 msgr2=0x7f3428071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 -- 172.21.15.148:0/4088010029 shutdown_connections 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.791+0000 7f3430659640 1 -- 172.21.15.148:0/4088010029 wait complete. 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.792+0000 7f3430659640 1 Processor -- start 2024-03-04T12:12:38.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.792+0000 7f3430659640 1 -- start start 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 0x7f3428155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 0x7f34280ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342806c550 con 0x7f342806bce0 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280cc3e0 con 0x7f342815bba0 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342815e0c0 con 0x7f342806c0c0 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59172/0 (socket says 172.21.15.148:59172) 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 -- 172.21.15.148:0/3886590801 learned_addr learned my addr 172.21.15.148:0/3886590801 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 -- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 msgr2=0x7f3428155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342dbf4640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 0x7f3428155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342ebf6640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 0x7f34280ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 0x7f3428155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 -- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 msgr2=0x7f34280ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 0x7f34280ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 -- 172.21.15.148:0/3886590801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428156310 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342ebf6640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 0x7f34280ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342dbf4640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 0x7f3428155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f342e3f5640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f342400c7f0 tx=0x7f342400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424015070 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3424004220 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34281565f0 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424007620 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.793+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3428156bb0 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.794+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3428074f60 con 0x7f342806bce0 2024-03-04T12:12:38.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.795+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f34240079d0 con 0x7f342806bce0 2024-03-04T12:12:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.795+0000 7f341effd640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 0x7f340c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.795+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f342408a930 con 0x7f342806bce0 2024-03-04T12:12:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.795+0000 7f342dbf4640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 0x7f340c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.796+0000 7f342dbf4640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 0x7f340c06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3418036000 tx=0x7f3418008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:38.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.800+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3424059440 con 0x7f342806bce0 2024-03-04T12:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.974+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34280af560 con 0x7f342806bce0 2024-03-04T12:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.974+0000 7f341effd640 1 -- 172.21.15.148:0/3886590801 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3424058d40 con 0x7f342806bce0 2024-03-04T12:12:38.976 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 msgr2=0x7f340c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 0x7f340c06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3418036000 tx=0x7f3418008030 comp rx=0 tx=0).stop 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 msgr2=0x7f34280cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f342400c7f0 tx=0x7f342400ccb0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 shutdown_connections 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f340c06b4d0 0x7f340c06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342815bba0 0x7f34280ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342806c0c0 0x7f3428155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 --2- 172.21.15.148:0/3886590801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342806bce0 0x7f34280cd590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:38.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 >> 172.21.15.148:0/3886590801 conn(0x7f3428071bc0 msgr2=0x7f342815fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:38.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 shutdown_connections 2024-03-04T12:12:38.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:38.976+0000 7f3430659640 1 -- 172.21.15.148:0/3886590801 wait complete. 2024-03-04T12:12:38.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:39.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 Processor -- start 2024-03-04T12:12:39.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 -- start start 2024-03-04T12:12:39.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4aac0cec40 0x7f4aac0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 0x7f4aac15d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0ca970 con 0x7f4aac0d3300 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac161d90 con 0x7f4aac0cec40 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.092+0000 7f4ab3197640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac162070 con 0x7f4aac0cf020 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab0f33640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4aac0cec40 0x7f4aac0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab1734640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4aac0d3300 0x7f4aac0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38288/0 (socket says 172.21.15.148:38288) 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab1734640 1 -- 172.21.15.148:0/3539186410 learned_addr learned my addr 172.21.15.148:0/3539186410 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4aabfff640 1 --2- 172.21.15.148:0/3539186410 >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 0x7f4aac15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab0f33640 1 -- 172.21.15.148:0/3539186410 >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 msgr2=0x7f4aac15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3539186410 >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 0x7f4aac15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab0f33640 1 -- 172.21.15.148:0/3539186410 >> v1:172.21.15.148:6789/0 conn(0x7f4aac0d3300 legacy=0x7f4aac0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4aab7fe640 1 -- 172.21.15.148:0/3539186410 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892718816 0 0) 0x7f4aac0ca970 con 0x7f4aac0d3300 2024-03-04T12:12:39.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4ab0f33640 1 -- 172.21.15.148:0/3539186410 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac162350 con 0x7f4aac0cec40 2024-03-04T12:12:39.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.093+0000 7f4aabfff640 1 --2- 172.21.15.148:0/3539186410 >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 0x7f4aac15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3539186410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0cec40 0x7f4aac0b6ea0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0d18d0 tx=0x7f4aa002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9be7757683c124 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4aab7fe640 1 -- 172.21.15.148:0/3539186410 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0035020 con 0x7f4aac0cec40 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4aab7fe640 1 -- 172.21.15.148:0/3539186410 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0031830 con 0x7f4aac0cec40 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4aab7fe640 1 -- 172.21.15.148:0/3539186410 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0031c00 con 0x7f4aac0cec40 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 -- 172.21.15.148:0/3539186410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0cec40 msgr2=0x7f4aac0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3539186410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0cec40 0x7f4aac0b6ea0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0d18d0 tx=0x7f4aa002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 -- 172.21.15.148:0/3539186410 shutdown_connections 2024-03-04T12:12:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3539186410 >> v2:172.21.15.191:3300/0 conn(0x7f4aac0cf020 0x7f4aac15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3539186410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0cec40 0x7f4aac0b6ea0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 -- 172.21.15.148:0/3539186410 >> 172.21.15.148:0/3539186410 conn(0x7f4aac0d0580 msgr2=0x7f4aac0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 -- 172.21.15.148:0/3539186410 shutdown_connections 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.094+0000 7f4ab3197640 1 -- 172.21.15.148:0/3539186410 wait complete. 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 Processor -- start 2024-03-04T12:12:39.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 -- start start 2024-03-04T12:12:39.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 0x7f4aac0aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 0x7f4aac1600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1617d0 con 0x7f4aac0d3300 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0d1bf0 con 0x7f4aac0cec40 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.096+0000 7f4ab3197640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac161950 con 0x7f4aac0cf020 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59186/0 (socket says 172.21.15.148:59186) 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 -- 172.21.15.148:0/3465848953 learned_addr learned my addr 172.21.15.148:0/3465848953 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 0x7f4aac0aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4aabfff640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 0x7f4aac1600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 -- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 msgr2=0x7f4aac1600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 0x7f4aac1600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 -- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 msgr2=0x7f4aac0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 0x7f4aac0aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 -- 172.21.15.148:0/3465848953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0aa9e0 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 0x7f4aac0aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4aabfff640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 0x7f4aac1600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab1734640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c700 tx=0x7f4aa400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4018070 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa400ce10 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac0aaca0 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400a880 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.097+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac0ab260 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.098+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac074f60 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.099+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4aa401b070 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.099+0000 7f4aa97fa640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.099+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4aa4089b00 con 0x7f4aac0d3300 2024-03-04T12:12:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.099+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.099+0000 7f4ab0f33640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 0x7f4a9006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0036000 tx=0x7f4aa0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.103+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aa4058610 con 0x7f4aac0d3300 2024-03-04T12:12:39.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.274+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4aac0d1930 con 0x7f4aac0d3300 2024-03-04T12:12:39.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.274+0000 7f4aa97fa640 1 -- 172.21.15.148:0/3465848953 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4aa4057f10 con 0x7f4aac0d3300 2024-03-04T12:12:39.276 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 msgr2=0x7f4a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 0x7f4a9006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0036000 tx=0x7f4aa0008030 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 msgr2=0x7f4aac158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c700 tx=0x7f4aa400cbc0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 shutdown_connections 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4aac0d3300 0x7f4aac158b80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aac0cf020 0x7f4aac1600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 --2- 172.21.15.148:0/3465848953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aac0cec40 0x7f4aac0aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 >> 172.21.15.148:0/3465848953 conn(0x7f4aac0d0580 msgr2=0x7f4aac0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 shutdown_connections 2024-03-04T12:12:39.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.276+0000 7f4ab3197640 1 -- 172.21.15.148:0/3465848953 wait complete. 2024-03-04T12:12:39.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:39.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.391+0000 7f99444d1640 1 Processor -- start 2024-03-04T12:12:39.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.391+0000 7f99444d1640 1 -- start start 2024-03-04T12:12:39.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f99444d1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f993c1450e0 0x7f993c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f99444d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f993c15c9b0 0x7f993c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f99444d1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0cebd0 con 0x7f993c0cc340 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f99444d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0cc7b0 con 0x7f993c15c9b0 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f99444d1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0cf7e0 con 0x7f993c1450e0 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f993c15c9b0 0x7f993c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f993c15c9b0 0x7f993c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59198/0 (socket says 172.21.15.148:59198) 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 -- 172.21.15.148:0/1276026903 learned_addr learned my addr 172.21.15.148:0/1276026903 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f994226d640 1 --2- 172.21.15.148:0/1276026903 >> v2:172.21.15.191:3300/0 conn(0x7f993c1450e0 0x7f993c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 -- 172.21.15.148:0/1276026903 >> v2:172.21.15.191:3300/0 conn(0x7f993c1450e0 msgr2=0x7f993c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 --2- 172.21.15.148:0/1276026903 >> v2:172.21.15.191:3300/0 conn(0x7f993c1450e0 0x7f993c0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 -- 172.21.15.148:0/1276026903 >> v1:172.21.15.191:6789/0 conn(0x7f993c0cc340 legacy=0x7f993c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.392+0000 7f9941a6c640 1 -- 172.21.15.148:0/1276026903 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0ceeb0 con 0x7f993c15c9b0 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f9941a6c640 1 --2- 172.21.15.148:0/1276026903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c15c9b0 0x7f993c0cbe00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f993c0b23d0 tx=0x7f992c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf74045bb5fe8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f994126b640 1 -- 172.21.15.148:0/1276026903 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c035020 con 0x7f993c15c9b0 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f994126b640 1 -- 172.21.15.148:0/1276026903 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c031870 con 0x7f993c15c9b0 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f994126b640 1 -- 172.21.15.148:0/1276026903 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c031c40 con 0x7f993c15c9b0 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f99444d1640 1 -- 172.21.15.148:0/1276026903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c15c9b0 msgr2=0x7f993c0cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.393+0000 7f99444d1640 1 --2- 172.21.15.148:0/1276026903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c15c9b0 0x7f993c0cbe00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f993c0b23d0 tx=0x7f992c02b910 comp rx=0 tx=0).stop 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 -- 172.21.15.148:0/1276026903 shutdown_connections 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 --2- 172.21.15.148:0/1276026903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c15c9b0 0x7f993c0cbe00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 --2- 172.21.15.148:0/1276026903 >> v2:172.21.15.191:3300/0 conn(0x7f993c1450e0 0x7f993c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 -- 172.21.15.148:0/1276026903 >> 172.21.15.148:0/1276026903 conn(0x7f993c06acd0 msgr2=0x7f993c06b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 -- 172.21.15.148:0/1276026903 shutdown_connections 2024-03-04T12:12:39.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.394+0000 7f99444d1640 1 -- 172.21.15.148:0/1276026903 wait complete. 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 Processor -- start 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 -- start start 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 0x7f993c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f993c15c9b0 0x7f993c13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c15ce60 con 0x7f993c1450e0 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0cf470 con 0x7f993c0cc340 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.395+0000 7f99444d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1454c0 con 0x7f993c15c9b0 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59208/0 (socket says 172.21.15.148:59208) 2024-03-04T12:12:39.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 -- 172.21.15.148:0/1320256187 learned_addr learned my addr 172.21.15.148:0/1320256187 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 -- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f993c15c9b0 msgr2=0x7f993c13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9942a6e640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f993c15c9b0 0x7f993c13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f993c15c9b0 0x7f993c13c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f994226d640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 0x7f993c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 -- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 msgr2=0x7f993c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 0x7f993c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 -- 172.21.15.148:0/1320256187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c13daf0 con 0x7f993c1450e0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f994226d640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 0x7f993c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9941a6c640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c040070 con 0x7f993c1450e0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c030d00 con 0x7f993c1450e0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c13ea50 con 0x7f993c1450e0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c035040 con 0x7f993c1450e0 2024-03-04T12:12:39.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.396+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c15d880 con 0x7f993c1450e0 2024-03-04T12:12:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.397+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c0ce340 con 0x7f993c1450e0 2024-03-04T12:12:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.398+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f992c04b060 con 0x7f993c1450e0 2024-03-04T12:12:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.398+0000 7f9932ffd640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 0x7f992006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.398+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f992c0ac350 con 0x7f993c1450e0 2024-03-04T12:12:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.398+0000 7f994226d640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 0x7f992006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.398+0000 7f994226d640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 0x7f992006d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.402+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f993c0ce340 con 0x7f993c1450e0 2024-03-04T12:12:39.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.572+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f993c074f60 con 0x7f993c1450e0 2024-03-04T12:12:39.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.572+0000 7f9932ffd640 1 -- 172.21.15.148:0/1320256187 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f992c07ae60 con 0x7f993c1450e0 2024-03-04T12:12:39.574 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 msgr2=0x7f992006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 0x7f992006d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938008040 comp rx=0 tx=0).stop 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 msgr2=0x7f993c14ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 shutdown_connections 2024-03-04T12:12:39.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f992006b4d0 0x7f992006d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f993c15c9b0 0x7f993c13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f993c1450e0 0x7f993c14ba70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 --2- 172.21.15.148:0/1320256187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f993c0cc340 0x7f993c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 >> 172.21.15.148:0/1320256187 conn(0x7f993c06acd0 msgr2=0x7f993c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 shutdown_connections 2024-03-04T12:12:39.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.574+0000 7f99444d1640 1 -- 172.21.15.148:0/1320256187 wait complete. 2024-03-04T12:12:39.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.686+0000 7f5417190640 1 Processor -- start 2024-03-04T12:12:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.686+0000 7f5417190640 1 -- start start 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f5417190640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54100cfae0 0x7f54100714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f5417190640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541015f020 con 0x7f54100cf650 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f5417190640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100cd010 con 0x7f5410154a80 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f5417190640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54100cff70 con 0x7f54100cfae0 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f541572d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54100cfae0 0x7f54100714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f540ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54100cf650 0x7f5410070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38294/0 (socket says 172.21.15.148:38294) 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f540ffff640 1 -- 172.21.15.148:0/1091413451 learned_addr learned my addr 172.21.15.148:0/1091413451 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f541572d640 1 -- 172.21.15.148:0/1091413451 >> v1:172.21.15.191:6789/0 conn(0x7f5410154a80 legacy=0x7f541015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f541572d640 1 -- 172.21.15.148:0/1091413451 >> v1:172.21.15.148:6789/0 conn(0x7f54100cf650 legacy=0x7f5410070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f540f7fe640 1 -- 172.21.15.148:0/1091413451 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865519048 0 0) 0x7f53fc001110 con 0x7f54100cf650 2024-03-04T12:12:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.687+0000 7f541572d640 1 -- 172.21.15.148:0/1091413451 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100cdaa0 con 0x7f54100cfae0 2024-03-04T12:12:39.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f541572d640 1 --2- 172.21.15.148:0/1091413451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cfae0 0x7f54100714f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5408009d90 tx=0x7f540802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bb632e73ef38d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f540f7fe640 1 -- 172.21.15.148:0/1091413451 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408039070 con 0x7f54100cfae0 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f540f7fe640 1 -- 172.21.15.148:0/1091413451 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540802cd00 con 0x7f54100cfae0 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f540f7fe640 1 -- 172.21.15.148:0/1091413451 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408034660 con 0x7f54100cfae0 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f5417190640 1 -- 172.21.15.148:0/1091413451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cfae0 msgr2=0x7f54100714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.688+0000 7f5417190640 1 --2- 172.21.15.148:0/1091413451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cfae0 0x7f54100714f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5408009d90 tx=0x7f540802bf50 comp rx=0 tx=0).stop 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.689+0000 7f5417190640 1 -- 172.21.15.148:0/1091413451 shutdown_connections 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.689+0000 7f5417190640 1 --2- 172.21.15.148:0/1091413451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cfae0 0x7f54100714f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.689+0000 7f5417190640 1 -- 172.21.15.148:0/1091413451 >> 172.21.15.148:0/1091413451 conn(0x7f541006f5f0 msgr2=0x7f541006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.689+0000 7f5417190640 1 -- 172.21.15.148:0/1091413451 shutdown_connections 2024-03-04T12:12:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.689+0000 7f5417190640 1 -- 172.21.15.148:0/1091413451 wait complete. 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.690+0000 7f5417190640 1 Processor -- start 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.690+0000 7f5417190640 1 -- start start 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54100cfae0 0x7f54100c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5410154a80 0x7f54101f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100ce340 con 0x7f54100cf650 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100ce060 con 0x7f5410154a80 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54100ce620 con 0x7f54100cfae0 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59222/0 (socket says 172.21.15.148:59222) 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 -- 172.21.15.148:0/3773593019 learned_addr learned my addr 172.21.15.148:0/3773593019 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 -- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54100cfae0 msgr2=0x7f54100c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54100cfae0 0x7f54100c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 -- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5410154a80 msgr2=0x7f54101f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5410154a80 0x7f54101f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 -- 172.21.15.148:0/3773593019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100b5f70 con 0x7f54100cf650 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5414f2c640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f540400c400 tx=0x7f540400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404017070 con 0x7f54100cf650 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54040052d0 con 0x7f54100cf650 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54100b6250 con 0x7f54100cf650 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404003830 con 0x7f54100cf650 2024-03-04T12:12:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.691+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54100b6810 con 0x7f54100cf650 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.692+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54100b3150 con 0x7f54100cf650 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.693+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f540401a070 con 0x7f54100cf650 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.693+0000 7f540d7fa640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 0x7f53f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.693+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5404089950 con 0x7f54100cf650 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.693+0000 7f540ffff640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 0x7f53f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.694+0000 7f540ffff640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 0x7f53f406d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0013b0 tx=0x7f53fc001300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.697+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54100b3150 con 0x7f54100cf650 2024-03-04T12:12:39.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.869+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5410074f60 con 0x7f54100cf650 2024-03-04T12:12:39.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.870+0000 7f540d7fa640 1 -- 172.21.15.148:0/3773593019 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f54040583d0 con 0x7f54100cf650 2024-03-04T12:12:39.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:39.872 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 msgr2=0x7f53f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 0x7f53f406d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0013b0 tx=0x7f53fc001300 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 msgr2=0x7f54100b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f540400c400 tx=0x7f540400c8c0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 shutdown_connections 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f53f406b500 0x7f53f406d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5410154a80 0x7f54101f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54100cfae0 0x7f54100c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 --2- 172.21.15.148:0/3773593019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100cf650 0x7f54100b5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 >> 172.21.15.148:0/3773593019 conn(0x7f541006f5f0 msgr2=0x7f541006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 shutdown_connections 2024-03-04T12:12:39.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.872+0000 7f5417190640 1 -- 172.21.15.148:0/3773593019 wait complete. 2024-03-04T12:12:39.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:39.886 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.988+0000 7fd85cddf640 1 Processor -- start 2024-03-04T12:12:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.988+0000 7fd85cddf640 1 -- start start 2024-03-04T12:12:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 0x7fd8580ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8580cf660 0x7fd8580cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 0x7fd858161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8580d0000 con 0x7fd8580cf660 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8580cc920 con 0x7fd85815da20 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85cddf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd858162450 con 0x7fd8581616e0 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8580cf660 0x7fd8580cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8580cf660 0x7fd8580cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59238/0 (socket says 172.21.15.148:59238) 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 -- 172.21.15.148:0/3059517928 learned_addr learned my addr 172.21.15.148:0/3059517928 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85659c640 1 --2- 172.21.15.148:0/3059517928 >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 0x7fd8580ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd856d9d640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 0x7fd858161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 -- 172.21.15.148:0/3059517928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 msgr2=0x7fd858161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 0x7fd858161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 -- 172.21.15.148:0/3059517928 >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 msgr2=0x7fd8580ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 --2- 172.21.15.148:0/3059517928 >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 0x7fd8580ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd84ffff640 1 -- 172.21.15.148:0/3059517928 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580cc2d0 con 0x7fd8580cf660 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd856d9d640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 0x7fd858161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.989+0000 7fd85659c640 1 --2- 172.21.15.148:0/3059517928 >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 0x7fd8580ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.990+0000 7fd84ffff640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8580cf660 0x7fd8580cfac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd8580cf310 tx=0x7fd84002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da09b03ffc09dab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.990+0000 7fd855d9b640 1 -- 172.21.15.148:0/3059517928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840035020 con 0x7fd8580cf660 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.990+0000 7fd855d9b640 1 -- 172.21.15.148:0/3059517928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840031870 con 0x7fd8580cf660 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.990+0000 7fd855d9b640 1 -- 172.21.15.148:0/3059517928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840031c40 con 0x7fd8580cf660 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 -- 172.21.15.148:0/3059517928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8580cf660 msgr2=0x7fd8580cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8580cf660 0x7fd8580cfac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd8580cf310 tx=0x7fd84002b910 comp rx=0 tx=0).stop 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 -- 172.21.15.148:0/3059517928 shutdown_connections 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8581616e0 0x7fd858161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 --2- 172.21.15.148:0/3059517928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8580cf660 0x7fd8580cfac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 --2- 172.21.15.148:0/3059517928 >> v2:172.21.15.191:3300/0 conn(0x7fd85815da20 0x7fd8580ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 -- 172.21.15.148:0/3059517928 >> 172.21.15.148:0/3059517928 conn(0x7fd8580b33a0 msgr2=0x7fd8580b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 -- 172.21.15.148:0/3059517928 shutdown_connections 2024-03-04T12:12:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.991+0000 7fd85cddf640 1 -- 172.21.15.148:0/3059517928 wait complete. 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.992+0000 7fd85cddf640 1 Processor -- start 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.992+0000 7fd85cddf640 1 -- start start 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8580cf660 0x7fd858151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd85815da20 0x7fd858152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580cbf80 con 0x7fd8581616e0 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580cd2f0 con 0x7fd85815da20 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85815de00 con 0x7fd8580cf660 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59248/0 (socket says 172.21.15.148:59248) 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 -- 172.21.15.148:0/1033164062 learned_addr learned my addr 172.21.15.148:0/1033164062 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85659c640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8580cf660 0x7fd858151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd84ffff640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd85815da20 0x7fd858152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 -- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8580cf660 msgr2=0x7fd858151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8580cf660 0x7fd858151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 -- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd85815da20 msgr2=0x7fd858152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd85815da20 0x7fd858152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 -- 172.21.15.148:0/1033164062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85815f390 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd856d9d640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd85000cd50 tx=0x7fd85000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000f680 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000f840 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85815f670 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.993+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000fc30 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.994+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85815fc30 con 0x7fd8581616e0 2024-03-04T12:12:39.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.995+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8581559b0 con 0x7fd8581616e0 2024-03-04T12:12:39.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.995+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd8500213f0 con 0x7fd8581616e0 2024-03-04T12:12:39.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.995+0000 7fd84effd640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:39.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.995+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd85008b190 con 0x7fd8581616e0 2024-03-04T12:12:39.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.995+0000 7fd85659c640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:39.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:39.996+0000 7fd85659c640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 0x7fd83806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd84800ab60 tx=0x7fd848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.000+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd850014030 con 0x7fd8581616e0 2024-03-04T12:12:40.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.174+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd85806b800 con 0x7fd8581616e0 2024-03-04T12:12:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.174+0000 7fd84effd640 1 -- 172.21.15.148:0/1033164062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd85806b800 con 0x7fd8581616e0 2024-03-04T12:12:40.176 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 msgr2=0x7fd83806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 0x7fd83806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd84800ab60 tx=0x7fd848008040 comp rx=0 tx=0).stop 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 msgr2=0x7fd8580c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd85000cd50 tx=0x7fd85000a650 comp rx=0 tx=0).stop 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 shutdown_connections 2024-03-04T12:12:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8581616e0 0x7fd8580c7d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd85815da20 0x7fd858152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 --2- 172.21.15.148:0/1033164062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8580cf660 0x7fd858151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 >> 172.21.15.148:0/1033164062 conn(0x7fd8580b33a0 msgr2=0x7fd85806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 shutdown_connections 2024-03-04T12:12:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.176+0000 7fd85cddf640 1 -- 172.21.15.148:0/1033164062 wait complete. 2024-03-04T12:12:40.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:40.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.291+0000 7f3f319fa640 1 Processor -- start 2024-03-04T12:12:40.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.291+0000 7f3f319fa640 1 -- start start 2024-03-04T12:12:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f319fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f2c15bca0 0x7f3f2c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f319fa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c066bd0 con 0x7f3f2c0cf2d0 2024-03-04T12:12:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f319fa640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c0cec00 con 0x7f3f2c0af420 2024-03-04T12:12:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f319fa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c06b1e0 con 0x7f3f2c15bca0 2024-03-04T12:12:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f2a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f2c15bca0 0x7f3f2c0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f2affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3f2c0cf2d0 0x7f3f2c0cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38300/0 (socket says 172.21.15.148:38300) 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f2a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f2c15bca0 0x7f3f2c0cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59258/0 (socket says 172.21.15.148:59258) 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.292+0000 7f3f2affd640 1 -- 172.21.15.148:0/1316100320 learned_addr learned my addr 172.21.15.148:0/1316100320 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.293+0000 7f3f2a7fc640 1 -- 172.21.15.148:0/1316100320 >> v1:172.21.15.191:6789/0 conn(0x7f3f2c0af420 legacy=0x7f3f2c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.293+0000 7f3f2a7fc640 1 -- 172.21.15.148:0/1316100320 >> v1:172.21.15.148:6789/0 conn(0x7f3f2c0cf2d0 legacy=0x7f3f2c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.293+0000 7f3f2a7fc640 1 -- 172.21.15.148:0/1316100320 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0ceee0 con 0x7f3f2c15bca0 2024-03-04T12:12:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.293+0000 7f3f2a7fc640 1 --2- 172.21.15.148:0/1316100320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c15bca0 0x7f3f2c0cd490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6797da4ca922b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.293+0000 7f3f29ffb640 1 -- 172.21.15.148:0/1316100320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200315a0 con 0x7f3f2c15bca0 2024-03-04T12:12:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f29ffb640 1 -- 172.21.15.148:0/1316100320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20037070 con 0x7f3f2c15bca0 2024-03-04T12:12:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f29ffb640 1 -- 172.21.15.148:0/1316100320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20031960 con 0x7f3f2c15bca0 2024-03-04T12:12:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 -- 172.21.15.148:0/1316100320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c15bca0 msgr2=0x7f3f2c0cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1316100320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c15bca0 0x7f3f2c0cd490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002b840 comp rx=0 tx=0).stop 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 -- 172.21.15.148:0/1316100320 shutdown_connections 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1316100320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c15bca0 0x7f3f2c0cd490 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 -- 172.21.15.148:0/1316100320 >> 172.21.15.148:0/1316100320 conn(0x7f3f2c0cc970 msgr2=0x7f3f2c0ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 -- 172.21.15.148:0/1316100320 shutdown_connections 2024-03-04T12:12:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.294+0000 7f3f319fa640 1 -- 172.21.15.148:0/1316100320 wait complete. 2024-03-04T12:12:40.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.295+0000 7f3f319fa640 1 Processor -- start 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.295+0000 7f3f319fa640 1 -- start start 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f2c0cf2d0 0x7f3f2c0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f2c155940 0x7f3f2c154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c09e450 con 0x7f3f2c0af420 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c09e2d0 con 0x7f3f2c155940 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f319fa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c06bd80 con 0x7f3f2c0cf2d0 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f2affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f2affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59262/0 (socket says 172.21.15.148:59262) 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f2affd640 1 -- 172.21.15.148:0/1098904214 learned_addr learned my addr 172.21.15.148:0/1098904214 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f2a7fc640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f2c0cf2d0 0x7f3f2c0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.296+0000 7f3f2b7fe640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f2c155940 0x7f3f2c154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 -- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f2c0cf2d0 msgr2=0x7f3f2c0c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f2c0cf2d0 0x7f3f2c0c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 -- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f2c155940 msgr2=0x7f3f2c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f2c155940 0x7f3f2c154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 -- 172.21.15.148:0/1098904214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0c45d0 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f2affd640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00cb10 tx=0x7f3f1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c017070 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c00ac70 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c0c48b0 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c012620 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.297+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c0c4e70 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.298+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c074f60 con 0x7f3f2c0af420 2024-03-04T12:12:40.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.298+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3f1c01a070 con 0x7f3f2c0af420 2024-03-04T12:12:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.299+0000 7f3f13fff640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 0x7f3f1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.299+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3f1c08a430 con 0x7f3f2c0af420 2024-03-04T12:12:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.299+0000 7f3f2a7fc640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 0x7f3f1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.299+0000 7f3f2a7fc640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 0x7f3f1806d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f20036000 tx=0x7f3f20008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.303+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f1c058e20 con 0x7f3f2c0af420 2024-03-04T12:12:40.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.475+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f2c06b710 con 0x7f3f2c0af420 2024-03-04T12:12:40.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.475+0000 7f3f13fff640 1 -- 172.21.15.148:0/1098904214 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f1c058720 con 0x7f3f2c0af420 2024-03-04T12:12:40.477 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:40.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 msgr2=0x7f3f1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 0x7f3f1806d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f20036000 tx=0x7f3f20008030 comp rx=0 tx=0).stop 2024-03-04T12:12:40.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 msgr2=0x7f3f2c0c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00cb10 tx=0x7f3f1c00a060 comp rx=0 tx=0).stop 2024-03-04T12:12:40.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 shutdown_connections 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3f1806b500 0x7f3f1806d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f2c155940 0x7f3f2c154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f2c0cf2d0 0x7f3f2c0c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 --2- 172.21.15.148:0/1098904214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0af420 0x7f3f2c0c5670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.477+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 >> 172.21.15.148:0/1098904214 conn(0x7f3f2c0cc970 msgr2=0x7f3f2c15ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.478+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 shutdown_connections 2024-03-04T12:12:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.478+0000 7f3f319fa640 1 -- 172.21.15.148:0/1098904214 wait complete. 2024-03-04T12:12:40.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.593+0000 7fbfe757e640 1 Processor -- start 2024-03-04T12:12:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.593+0000 7fbfe757e640 1 -- start start 2024-03-04T12:12:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 0x7fbfe00b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cf020 0x7fbfe015d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe00d3300 0x7fbfe015dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00ca970 con 0x7fbfe00d3300 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe0161d90 con 0x7fbfe00cec40 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe757e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0162070 con 0x7fbfe00cf020 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe5b1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe00d3300 0x7fbfe015dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe5b1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe00d3300 0x7fbfe015dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59268/0 (socket says 172.21.15.148:59268) 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe5b1b640 1 -- 172.21.15.148:0/686844408 learned_addr learned my addr 172.21.15.148:0/686844408 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe4b19640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cf020 0x7fbfe015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.594+0000 7fbfe531a640 1 --2- 172.21.15.148:0/686844408 >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 0x7fbfe00b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 -- 172.21.15.148:0/686844408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cf020 msgr2=0x7fbfe015d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cf020 0x7fbfe015d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 -- 172.21.15.148:0/686844408 >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 msgr2=0x7fbfe00b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 --2- 172.21.15.148:0/686844408 >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 0x7fbfe00b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 -- 172.21.15.148:0/686844408 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0162350 con 0x7fbfe00d3300 2024-03-04T12:12:40.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe531a640 1 --2- 172.21.15.148:0/686844408 >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 0x7fbfe00b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfe5b1b640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00d3300 0x7fbfe015dc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00674a0 tx=0x7fbfd802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739050dac1192f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfd7fff640 1 -- 172.21.15.148:0/686844408 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd802ca70 con 0x7fbfe00d3300 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfd7fff640 1 -- 172.21.15.148:0/686844408 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd802cc10 con 0x7fbfe00d3300 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.595+0000 7fbfd7fff640 1 -- 172.21.15.148:0/686844408 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80343f0 con 0x7fbfe00d3300 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 -- 172.21.15.148:0/686844408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00d3300 msgr2=0x7fbfe015dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00d3300 0x7fbfe015dc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00674a0 tx=0x7fbfd802bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 -- 172.21.15.148:0/686844408 shutdown_connections 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cf020 0x7fbfe015d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 --2- 172.21.15.148:0/686844408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00d3300 0x7fbfe015dc70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 --2- 172.21.15.148:0/686844408 >> v2:172.21.15.191:3300/0 conn(0x7fbfe00cec40 0x7fbfe00b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 -- 172.21.15.148:0/686844408 >> 172.21.15.148:0/686844408 conn(0x7fbfe00d0580 msgr2=0x7fbfe00d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 -- 172.21.15.148:0/686844408 shutdown_connections 2024-03-04T12:12:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.596+0000 7fbfe757e640 1 -- 172.21.15.148:0/686844408 wait complete. 2024-03-04T12:12:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.597+0000 7fbfe757e640 1 Processor -- start 2024-03-04T12:12:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.597+0000 7fbfe757e640 1 -- start start 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cec40 0x7fbfe00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 0x7fbfe014c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01617d0 con 0x7fbfe00cf020 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00d1bf0 con 0x7fbfe00d3300 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe757e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0161950 con 0x7fbfe00cec40 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59282/0 (socket says 172.21.15.148:59282) 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 -- 172.21.15.148:0/3111900385 learned_addr learned my addr 172.21.15.148:0/3111900385 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe5b1b640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 0x7fbfe014c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe531a640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cec40 0x7fbfe00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 -- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cec40 msgr2=0x7fbfe00c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cec40 0x7fbfe00c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 -- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 msgr2=0x7fbfe014c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 0x7fbfe014c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 -- 172.21.15.148:0/3111900385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00c99e0 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe5b1b640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 0x7fbfe014c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.598+0000 7fbfe4b19640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9e0 tx=0x7fbfd000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.599+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0017070 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.599+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000c9d0 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.599+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe01f72a0 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.599+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000cdc0 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.599+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01f7860 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.600+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0074f60 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.600+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbfd001a070 con 0x7fbfe00cf020 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.601+0000 7fbfd5ffb640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.601+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbfd008a5b0 con 0x7fbfe00cf020 2024-03-04T12:12:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.601+0000 7fbfe531a640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.601+0000 7fbfe531a640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 0x7fbfc406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006fd0 tx=0x7fbfdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.605+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfd0059030 con 0x7fbfe00cf020 2024-03-04T12:12:40.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.775+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfe00d1930 con 0x7fbfe00cf020 2024-03-04T12:12:40.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.775+0000 7fbfd5ffb640 1 -- 172.21.15.148:0/3111900385 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfd0058930 con 0x7fbfe00cf020 2024-03-04T12:12:40.777 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 msgr2=0x7fbfc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 0x7fbfc406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc006fd0 tx=0x7fbfdc008040 comp rx=0 tx=0).stop 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 msgr2=0x7fbfe015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9e0 tx=0x7fbfd000eea0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 shutdown_connections 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe00d3300 0x7fbfe014c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe00cf020 0x7fbfe015c250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 --2- 172.21.15.148:0/3111900385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe00cec40 0x7fbfe00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 >> 172.21.15.148:0/3111900385 conn(0x7fbfe00d0580 msgr2=0x7fbfe0157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 shutdown_connections 2024-03-04T12:12:40.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.777+0000 7fbfe757e640 1 -- 172.21.15.148:0/3111900385 wait complete. 2024-03-04T12:12:40.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:40.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 Processor -- start 2024-03-04T12:12:40.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 -- start start 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe874168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740cf740 con 0x7fe874161390 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87406c7d0 con 0x7fe87406c060 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.893+0000 7fe87a390640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406d3e0 con 0x7fe8740cedf0 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe873fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe874161390 0x7fe8740d20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38308/0 (socket says 172.21.15.148:38308) 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe873fff640 1 -- 172.21.15.148:0/742817729 learned_addr learned my addr 172.21.15.148:0/742817729 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717799009 0 0) 0x7fe8740cf740 con 0x7fe874161390 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe87892d640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe874168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c003520 con 0x7fe874161390 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441701967 0 0) 0x7fe85c003520 con 0x7fe874161390 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 msgr2=0x7fe874168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe874168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 >> v1:172.21.15.191:6789/0 conn(0x7fe87406c060 legacy=0x7fe8740ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87406cab0 con 0x7fe874161390 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fe864001670 con 0x7fe874161390 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 >> v1:172.21.15.148:6789/0 conn(0x7fe874161390 legacy=0x7fe8740d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe85c0039e0 0x7fe85c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe85c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85c004d40 0x7fe85c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406d3e0 con 0x7fe85c004d40 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c00baa0 con 0x7fe85c0039e0 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c00bca0 con 0x7fe8740cedf0 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe87892d640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85c004d40 0x7fe85c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe873fff640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe85c0039e0 0x7fe85c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.894+0000 7fe8737fe640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe85c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 -- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 msgr2=0x7fe85c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8740cedf0 0x7fe85c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 -- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe85c0039e0 msgr2=0x7fe85c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe85c0039e0 0x7fe85c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 -- 172.21.15.148:0/742817729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c003520 con 0x7fe85c004d40 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe87892d640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85c004d40 0x7fe85c0051c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009f90 tx=0x7fe85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d3571a52aa1847 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c039070 con 0x7fe85c004d40 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c02cbd0 con 0x7fe85c004d40 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.895+0000 7fe872ffd640 1 -- 172.21.15.148:0/742817729 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c009c10 con 0x7fe85c004d40 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 -- 172.21.15.148:0/742817729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85c004d40 msgr2=0x7fe85c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 --2- 172.21.15.148:0/742817729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85c004d40 0x7fe85c0051c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009f90 tx=0x7fe85c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87892d640 1 -- 172.21.15.148:0/742817729 reap_dead start 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 -- 172.21.15.148:0/742817729 shutdown_connections 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 -- 172.21.15.148:0/742817729 >> 172.21.15.148:0/742817729 conn(0x7fe874066dc0 msgr2=0x7fe8740671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:40.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 -- 172.21.15.148:0/742817729 shutdown_connections 2024-03-04T12:12:40.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.896+0000 7fe87a390640 1 -- 172.21.15.148:0/742817729 wait complete. 2024-03-04T12:12:40.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.897+0000 7fe87a390640 1 Processor -- start 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.897+0000 7fe87a390640 1 -- start start 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 0x7fe8740ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 0x7fe8740ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406c4c0 con 0x7fe87415f640 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406d070 con 0x7fe8740aeb50 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87a390640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe874161770 con 0x7fe8741f6fe0 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59300/0 (socket says 172.21.15.148:59300) 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 -- 172.21.15.148:0/2022966608 learned_addr learned my addr 172.21.15.148:0/2022966608 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87892d640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 0x7fe8740ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe873fff640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 0x7fe8740ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 -- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 msgr2=0x7fe8740ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 0x7fe8740ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 -- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 msgr2=0x7fe8740ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 0x7fe8740ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe8737fe640 1 -- 172.21.15.148:0/2022966608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87415fe00 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe873fff640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 0x7fe8740ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.898+0000 7fe87892d640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 0x7fe8740ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe8737fe640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe86800f820 tx=0x7fe86800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680050d0 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868018070 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741600e0 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868005490 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.899+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741606a0 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.900+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe874074f60 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.900+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe86800d6d0 con 0x7fe87415f640 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.901+0000 7fe870ff9640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 0x7fe86006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:40.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.901+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe86808a3c0 con 0x7fe87415f640 2024-03-04T12:12:40.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.901+0000 7fe873fff640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 0x7fe86006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:40.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.901+0000 7fe873fff640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 0x7fe86006d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe864001eb0 tx=0x7fe864009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:40.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:40.905+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe868058e40 con 0x7fe87415f640 2024-03-04T12:12:41.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.077+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8740aef30 con 0x7fe87415f640 2024-03-04T12:12:41.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.077+0000 7fe870ff9640 1 -- 172.21.15.148:0/2022966608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe868058740 con 0x7fe87415f640 2024-03-04T12:12:41.079 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:41.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 msgr2=0x7fe86006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 0x7fe86006d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe864001eb0 tx=0x7fe864009040 comp rx=0 tx=0).stop 2024-03-04T12:12:41.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 msgr2=0x7fe8740ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe86800f820 tx=0x7fe86800fce0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 shutdown_connections 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe86006b500 0x7fe86006d9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8740aeb50 0x7fe8740ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe87415f640 0x7fe8740ae610 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 --2- 172.21.15.148:0/2022966608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8741f6fe0 0x7fe8740ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.079+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 >> 172.21.15.148:0/2022966608 conn(0x7fe874066dc0 msgr2=0x7fe8740b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.080+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 shutdown_connections 2024-03-04T12:12:41.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.080+0000 7fe87a390640 1 -- 172.21.15.148:0/2022966608 wait complete. 2024-03-04T12:12:41.099 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-04T12:12:41.100 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:41.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.198+0000 7fdf60ca4640 1 Processor -- start 2024-03-04T12:12:41.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.198+0000 7fdf60ca4640 1 -- start start 2024-03-04T12:12:41.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf60ca4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf5c06d3b0 0x7fdf5c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf60ca4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf5c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf60ca4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c15be00 con 0x7fdf5c15b030 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf60ca4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0b2720 con 0x7fdf5c06d3b0 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf60ca4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06d8b0 con 0x7fdf5c15b410 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf5a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdf5c15b030 0x7fdf5c15dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38324/0 (socket says 172.21.15.148:38324) 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf5a59c640 1 -- 172.21.15.148:0/452156955 learned_addr learned my addr 172.21.15.148:0/452156955 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf5ad9d640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf5c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf59d9b640 1 --2- 172.21.15.148:0/452156955 >> v2:172.21.15.191:3300/0 conn(0x7fdf5c06d3b0 0x7fdf5c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.199+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511361347 0 0) 0x7fdf5c15be00 con 0x7fdf5c15b030 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf44003520 con 0x7fdf5c15b030 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957362633 0 0) 0x7fdf44003520 con 0x7fdf5c15b030 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 msgr2=0x7fdf5c15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf5c15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 >> v2:172.21.15.191:3300/0 conn(0x7fdf5c06d3b0 msgr2=0x7fdf5c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:41.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 --2- 172.21.15.148:0/452156955 >> v2:172.21.15.191:3300/0 conn(0x7fdf5c06d3b0 0x7fdf5c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c06db90 con 0x7fdf5c15b030 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fdf4c0019f0 con 0x7fdf5c15b030 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 >> v1:172.21.15.148:6789/0 conn(0x7fdf5c15b030 legacy=0x7fdf5c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf440039b0 0x7fdf44003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf44004800 0x7fdf44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0b2720 con 0x7fdf440039b0 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06d8b0 con 0x7fdf44004800 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4400b9f0 con 0x7fdf5c15b410 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf440039b0 0x7fdf44003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf59d9b640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf44004800 0x7fdf44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5ad9d640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 -- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 msgr2=0x7fdf440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:41.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c15b410 0x7fdf440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 -- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf44004800 msgr2=0x7fdf44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf44004800 0x7fdf44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.200+0000 7fdf5a59c640 1 -- 172.21.15.148:0/452156955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44003520 con 0x7fdf440039b0 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.201+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf440039b0 0x7fdf44003df0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00a870 tx=0x7fdf44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df94eebbd74fb36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.201+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c00a4c0 con 0x7fdf440039b0 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.201+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c034630 con 0x7fdf440039b0 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.201+0000 7fdf5959a640 1 -- 172.21.15.148:0/452156955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c034a20 con 0x7fdf440039b0 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 -- 172.21.15.148:0/452156955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf440039b0 msgr2=0x7fdf44003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/452156955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf440039b0 0x7fdf44003df0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00a870 tx=0x7fdf44005830 comp rx=0 tx=0).stop 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf5ad9d640 1 -- 172.21.15.148:0/452156955 reap_dead start 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 -- 172.21.15.148:0/452156955 shutdown_connections 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 -- 172.21.15.148:0/452156955 >> 172.21.15.148:0/452156955 conn(0x7fdf5c06acd0 msgr2=0x7fdf5c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 -- 172.21.15.148:0/452156955 shutdown_connections 2024-03-04T12:12:41.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.202+0000 7fdf60ca4640 1 -- 172.21.15.148:0/452156955 wait complete. 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.203+0000 7fdf60ca4640 1 Processor -- start 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.203+0000 7fdf60ca4640 1 -- start start 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 0x7fdf5c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 0x7fdf5c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06e690 con 0x7fdf5c0cd830 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06e150 con 0x7fdf440039b0 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf60ca4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06e810 con 0x7fdf5c0ac870 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59324/0 (socket says 172.21.15.148:59324) 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 -- 172.21.15.148:0/692770597 learned_addr learned my addr 172.21.15.148:0/692770597 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf5ad9d640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 0x7fdf5c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 0x7fdf5c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 -- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 msgr2=0x7fdf5c0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 0x7fdf5c0af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 -- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 msgr2=0x7fdf5c150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 0x7fdf5c150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf59d9b640 1 -- 172.21.15.148:0/692770597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c155d40 con 0x7fdf5c0cd830 2024-03-04T12:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf5ad9d640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 0x7fdf5c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.204+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 0x7fdf5c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf59d9b640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf50011960 tx=0x7fdf50011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50019070 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50004240 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000f880 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0acde0 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.205+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c0ad3a0 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.206+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c074f60 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.206+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf5001c070 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.206+0000 7fdf4affd640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 0x7fdf3c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.207+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdf5008a9e0 con 0x7fdf5c0cd830 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.207+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 0x7fdf3c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.207+0000 7fdf5a59c640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 0x7fdf3c06d960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c034000 tx=0x7fdf4c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.211+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf50059460 con 0x7fdf5c0cd830 2024-03-04T12:12:41.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.381+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf5c071c20 con 0x7fdf5c0cd830 2024-03-04T12:12:41.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.382+0000 7fdf4affd640 1 -- 172.21.15.148:0/692770597 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fdf50058d60 con 0x7fdf5c0cd830 2024-03-04T12:12:41.384 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:41.384 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:41.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 msgr2=0x7fdf3c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 0x7fdf3c06d960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c034000 tx=0x7fdf4c009030 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 msgr2=0x7fdf5c0af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf50011960 tx=0x7fdf50011e20 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 shutdown_connections 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdf3c06b4a0 0x7fdf3c06d960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5c0ac870 0x7fdf5c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cd830 0x7fdf5c0af490 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 --2- 172.21.15.148:0/692770597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440039b0 0x7fdf5c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 >> 172.21.15.148:0/692770597 conn(0x7fdf5c06acd0 msgr2=0x7fdf5c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 shutdown_connections 2024-03-04T12:12:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.385+0000 7fdf60ca4640 1 -- 172.21.15.148:0/692770597 wait complete. 2024-03-04T12:12:41.397 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:41.398 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:41.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.501+0000 7f662737e640 1 Processor -- start 2024-03-04T12:12:41.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.501+0000 7f662737e640 1 -- start start 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662737e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 0x7f66200cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662737e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662015cf50 con 0x7f662015c7f0 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662737e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66200cc390 con 0x7f66200b0150 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662737e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200cc670 con 0x7f662006b3e0 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662511a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f662015c7f0 0x7f662006acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38332/0 (socket says 172.21.15.148:38332) 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f662511a640 1 -- 172.21.15.148:0/1983440620 learned_addr learned my addr 172.21.15.148:0/1983440620 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6624919640 1 --2- 172.21.15.148:0/1983440620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 0x7f66200cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165244456 0 0) 0x7f662015cf50 con 0x7f662015c7f0 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f660c003520 con 0x7f662015c7f0 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250338978 0 0) 0x7f66200cc390 con 0x7f66200b0150 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6624919640 1 -- 172.21.15.148:0/1983440620 >> v1:172.21.15.191:6789/0 conn(0x7f66200b0150 legacy=0x7f66200cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6624919640 1 -- 172.21.15.148:0/1983440620 >> v1:172.21.15.148:6789/0 conn(0x7f662015c7f0 legacy=0x7f662006acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.502+0000 7f6624919640 1 -- 172.21.15.148:0/1983440620 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200cc950 con 0x7f662006b3e0 2024-03-04T12:12:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.503+0000 7f6624919640 1 --2- 172.21.15.148:0/1983440620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 0x7f66200cd150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66100097d0 tx=0x7f661002b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1425033fc35d06d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.503+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610037070 con 0x7f662006b3e0 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.503+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610030510 con 0x7f662006b3e0 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.503+0000 7f6617fff640 1 -- 172.21.15.148:0/1983440620 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0015f0 con 0x7f662006b3e0 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 -- 172.21.15.148:0/1983440620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 msgr2=0x7f66200cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 --2- 172.21.15.148:0/1983440620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 0x7f66200cd150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66100097d0 tx=0x7f661002b600 comp rx=0 tx=0).stop 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 -- 172.21.15.148:0/1983440620 shutdown_connections 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 --2- 172.21.15.148:0/1983440620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f662006b3e0 0x7f66200cd150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 -- 172.21.15.148:0/1983440620 >> 172.21.15.148:0/1983440620 conn(0x7f66200b22e0 msgr2=0x7f66200b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 -- 172.21.15.148:0/1983440620 shutdown_connections 2024-03-04T12:12:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.504+0000 7f662737e640 1 -- 172.21.15.148:0/1983440620 wait complete. 2024-03-04T12:12:41.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.505+0000 7f662737e640 1 Processor -- start 2024-03-04T12:12:41.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.505+0000 7f662737e640 1 -- start start 2024-03-04T12:12:41.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66200b0150 0x7f66200b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66200b42c0 0x7f66200b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66200cdda0 con 0x7f662006b3e0 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662015cbe0 con 0x7f66200b0150 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662737e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200cdf20 con 0x7f66200b42c0 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59332/0 (socket says 172.21.15.148:59332) 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 -- 172.21.15.148:0/3203780805 learned_addr learned my addr 172.21.15.148:0/3203780805 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662591b640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66200b42c0 0x7f66200b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f6624919640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66200b0150 0x7f66200b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 -- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66200b42c0 msgr2=0x7f66200b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66200b42c0 0x7f66200b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 -- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66200b0150 msgr2=0x7f66200b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66200b0150 0x7f66200b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.506+0000 7f662511a640 1 -- 172.21.15.148:0/3203780805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620157770 con 0x7f662006b3e0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f662511a640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c710 tx=0x7f661c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c018070 con 0x7f662006b3e0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c007500 con 0x7f662006b3e0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662014a100 con 0x7f662006b3e0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c004a20 con 0x7f662006b3e0 2024-03-04T12:12:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.507+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66200b4920 con 0x7f662006b3e0 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.508+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6620074f60 con 0x7f662006b3e0 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.508+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f661c01b070 con 0x7f662006b3e0 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.509+0000 7f6615ffb640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 0x7f660406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.509+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f661c089a90 con 0x7f662006b3e0 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.509+0000 7f6624919640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 0x7f660406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.509+0000 7f6624919640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 0x7f660406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6610034000 tx=0x7f6610008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.513+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f662014a100 con 0x7f662006b3e0 2024-03-04T12:12:41.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.682+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66200ca890 con 0x7f662006b3e0 2024-03-04T12:12:41.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.683+0000 7f6615ffb640 1 -- 172.21.15.148:0/3203780805 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6620157770 con 0x7f662006b3e0 2024-03-04T12:12:41.684 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 msgr2=0x7f660406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 0x7f660406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6610034000 tx=0x7f6610008030 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 msgr2=0x7f6620149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c710 tx=0x7f661c00cbd0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 shutdown_connections 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f660406b4d0 0x7f660406d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66200b42c0 0x7f66200b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66200b0150 0x7f66200b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 --2- 172.21.15.148:0/3203780805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006b3e0 0x7f6620149bc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 >> 172.21.15.148:0/3203780805 conn(0x7f66200b22e0 msgr2=0x7f66200b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 shutdown_connections 2024-03-04T12:12:41.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.685+0000 7f662737e640 1 -- 172.21.15.148:0/3203780805 wait complete. 2024-03-04T12:12:41.697 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.799+0000 7f3bd13df640 1 Processor -- start 2024-03-04T12:12:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.799+0000 7f3bd13df640 1 -- start start 2024-03-04T12:12:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bd13df640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 0x7f3bcc0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bd13df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bcc15de40 0x7f3bcc0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bd13df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0ca970 con 0x7f3bcc0cf020 2024-03-04T12:12:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bd13df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc161d90 con 0x7f3bcc15de40 2024-03-04T12:12:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bd13df640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc162070 con 0x7f3bcc0cec40 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bcc15de40 0x7f3bcc0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bca7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3bcc0cf020 0x7f3bcc15d730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38346/0 (socket says 172.21.15.148:38346) 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bca7fc640 1 -- 172.21.15.148:0/812780875 learned_addr learned my addr 172.21.15.148:0/812780875 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcb7fe640 1 -- 172.21.15.148:0/812780875 >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 msgr2=0x7f3bcc0b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcaffd640 1 --2- 172.21.15.148:0/812780875 >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 0x7f3bcc0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcb7fe640 1 --2- 172.21.15.148:0/812780875 >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 0x7f3bcc0b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcb7fe640 1 -- 172.21.15.148:0/812780875 >> v1:172.21.15.148:6789/0 conn(0x7f3bcc0cf020 legacy=0x7f3bcc15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcb7fe640 1 -- 172.21.15.148:0/812780875 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc162350 con 0x7f3bcc15de40 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bc9ffb640 1 -- 172.21.15.148:0/812780875 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477317564 0 0) 0x7f3bcc0ca970 con 0x7f3bcc0cf020 2024-03-04T12:12:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.800+0000 7f3bcaffd640 1 --2- 172.21.15.148:0/812780875 >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 0x7f3bcc0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.801+0000 7f3bcb7fe640 1 --2- 172.21.15.148:0/812780875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc15de40 0x7f3bcc0d2110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0674a0 tx=0x7f3bc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5ea7258ce7405d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.801+0000 7f3bc9ffb640 1 -- 172.21.15.148:0/812780875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4035020 con 0x7f3bcc15de40 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.801+0000 7f3bc9ffb640 1 -- 172.21.15.148:0/812780875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc4031830 con 0x7f3bcc15de40 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.801+0000 7f3bc9ffb640 1 -- 172.21.15.148:0/812780875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4031c00 con 0x7f3bcc15de40 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 -- 172.21.15.148:0/812780875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc15de40 msgr2=0x7f3bcc0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 --2- 172.21.15.148:0/812780875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc15de40 0x7f3bcc0d2110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0674a0 tx=0x7f3bc402b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 -- 172.21.15.148:0/812780875 shutdown_connections 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 --2- 172.21.15.148:0/812780875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc15de40 0x7f3bcc0d2110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 --2- 172.21.15.148:0/812780875 >> v2:172.21.15.191:3300/0 conn(0x7f3bcc0cec40 0x7f3bcc0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 -- 172.21.15.148:0/812780875 >> 172.21.15.148:0/812780875 conn(0x7f3bcc0d0580 msgr2=0x7f3bcc0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 -- 172.21.15.148:0/812780875 shutdown_connections 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.802+0000 7f3bd13df640 1 -- 172.21.15.148:0/812780875 wait complete. 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.803+0000 7f3bd13df640 1 Processor -- start 2024-03-04T12:12:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.803+0000 7f3bd13df640 1 -- start start 2024-03-04T12:12:41.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bcc0cf020 0x7f3bcc1e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 0x7f3bcc1e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0d3300 con 0x7f3bcc0cec40 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0d1bf0 con 0x7f3bcc15de40 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bd13df640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0d3480 con 0x7f3bcc0cf020 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59358/0 (socket says 172.21.15.148:59358) 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 -- 172.21.15.148:0/1852050887 learned_addr learned my addr 172.21.15.148:0/1852050887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bca7fc640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bcc0cf020 0x7f3bcc1e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcb7fe640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 0x7f3bcc1e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 -- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bcc0cf020 msgr2=0x7f3bcc1e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bcc0cf020 0x7f3bcc1e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 -- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 msgr2=0x7f3bcc1e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 0x7f3bcc1e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcaffd640 1 -- 172.21.15.148:0/1852050887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc159b90 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.804+0000 7f3bcb7fe640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 0x7f3bcc1e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bcaffd640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cf60 tx=0x7f3bbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc01a070 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00d840 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0cc910 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00dc10 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.805+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc0cced0 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.806+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc074f60 con 0x7f3bcc0cec40 2024-03-04T12:12:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.806+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3bbc01d070 con 0x7f3bcc0cec40 2024-03-04T12:12:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.807+0000 7f3bb37fe640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.807+0000 7f3bca7fc640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.807+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3bbc08ad50 con 0x7f3bcc0cec40 2024-03-04T12:12:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.807+0000 7f3bca7fc640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 0x7f3bb806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0d0c60 tx=0x7f3bc0007b90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:41.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.811+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bbc08e030 con 0x7f3bcc0cec40 2024-03-04T12:12:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.983+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bcc156900 con 0x7f3bcc0cec40 2024-03-04T12:12:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.984+0000 7f3bb37fe640 1 -- 172.21.15.148:0/1852050887 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bbc059210 con 0x7f3bcc0cec40 2024-03-04T12:12:41.985 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.985+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 msgr2=0x7f3bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.985+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 0x7f3bb806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0d0c60 tx=0x7f3bc0007b90 comp rx=0 tx=0).stop 2024-03-04T12:12:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.985+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 msgr2=0x7f3bcc0d4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.985+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cf60 tx=0x7f3bbc00cf90 comp rx=0 tx=0).stop 2024-03-04T12:12:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 shutdown_connections 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bcc15de40 0x7f3bcc1e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bcc0cf020 0x7f3bcc1e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 --2- 172.21.15.148:0/1852050887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bcc0cec40 0x7f3bcc0d4590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 >> 172.21.15.148:0/1852050887 conn(0x7f3bcc0d0580 msgr2=0x7f3bcc06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 shutdown_connections 2024-03-04T12:12:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:41.986+0000 7f3bd13df640 1 -- 172.21.15.148:0/1852050887 wait complete. 2024-03-04T12:12:41.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:42.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.101+0000 7ffad0e59640 1 Processor -- start 2024-03-04T12:12:42.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.101+0000 7ffad0e59640 1 -- start start 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffad0e59640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 0x7ffacc06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffad0e59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffad0e59640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc15bae0 con 0x7ffacc15b700 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffad0e59640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc06c680 con 0x7ffacc06bce0 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffad0e59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0adb90 con 0x7ffacc15e100 2024-03-04T12:12:42.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffacad9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffac9d9b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 0x7ffacc06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffaca59c640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ffacc15b700 0x7ffacc0ae020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:38550/0 (socket says 172.21.15.148:38550) 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffacad9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:52574/0 (socket says 172.21.15.148:52574) 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.102+0000 7ffaca59c640 1 -- 172.21.15.148:0/4244831880 learned_addr learned my addr 172.21.15.148:0/4244831880 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.103+0000 7ffacad9d640 1 -- 172.21.15.148:0/4244831880 >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 msgr2=0x7ffacc06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.103+0000 7ffacad9d640 1 --2- 172.21.15.148:0/4244831880 >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 0x7ffacc06c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.103+0000 7ffacad9d640 1 -- 172.21.15.148:0/4244831880 >> v1:172.21.15.191:6789/0 conn(0x7ffacc15b700 legacy=0x7ffacc0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.103+0000 7ffacad9d640 1 -- 172.21.15.148:0/4244831880 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc15ee60 con 0x7ffacc15e100 2024-03-04T12:12:42.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.103+0000 7ffac9d9b640 1 --2- 172.21.15.148:0/4244831880 >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 0x7ffacc06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:42.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffacad9d640 1 --2- 172.21.15.148:0/4244831880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffac40097a0 tx=0x7ffac402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dce25189a92e474 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffac959a640 1 -- 172.21.15.148:0/4244831880 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4035020 con 0x7ffacc15e100 2024-03-04T12:12:42.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffac959a640 1 -- 172.21.15.148:0/4244831880 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4031830 con 0x7ffacc15e100 2024-03-04T12:12:42.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffac959a640 1 -- 172.21.15.148:0/4244831880 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4031c00 con 0x7ffacc15e100 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 -- 172.21.15.148:0/4244831880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 msgr2=0x7ffacc15e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 --2- 172.21.15.148:0/4244831880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffac40097a0 tx=0x7ffac402b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 -- 172.21.15.148:0/4244831880 shutdown_connections 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 --2- 172.21.15.148:0/4244831880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15e100 0x7ffacc15e590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 --2- 172.21.15.148:0/4244831880 >> v2:172.21.15.191:3300/0 conn(0x7ffacc06bce0 0x7ffacc06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 -- 172.21.15.148:0/4244831880 >> 172.21.15.148:0/4244831880 conn(0x7ffacc071bc0 msgr2=0x7ffacc071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 -- 172.21.15.148:0/4244831880 shutdown_connections 2024-03-04T12:12:42.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.104+0000 7ffad0e59640 1 -- 172.21.15.148:0/4244831880 wait complete. 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 Processor -- start 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 -- start start 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc06bce0 0x7ffacc14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc15e100 0x7ffacc0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc15ead0 con 0x7ffacc15b700 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0ad820 con 0x7ffacc15e100 2024-03-04T12:12:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.106+0000 7ffad0e59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc15f2c0 con 0x7ffacc06bce0 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59362/0 (socket says 172.21.15.148:59362) 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 -- 172.21.15.148:0/561488521 learned_addr learned my addr 172.21.15.148:0/561488521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffacad9d640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc15e100 0x7ffacc0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffaca59c640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc06bce0 0x7ffacc14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 -- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc06bce0 msgr2=0x7ffacc14c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc06bce0 0x7ffacc14c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 -- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc15e100 msgr2=0x7ffacc0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc15e100 0x7ffacc0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 -- 172.21.15.148:0/561488521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc14c700 con 0x7ffacc15b700 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffac9d9b640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ebe0 tx=0x7ffac000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0017070 con 0x7ffacc15b700 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac000cc30 con 0x7ffacc15b700 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc0cbbf0 con 0x7ffacc15b700 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0013470 con 0x7ffacc15b700 2024-03-04T12:12:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.107+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc1f3cd0 con 0x7ffacc15b700 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.108+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc074f60 con 0x7ffacc15b700 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.109+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffac001a070 con 0x7ffacc15b700 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.109+0000 7ffabaffd640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 0x7ffaac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.109+0000 7ffaca59c640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 0x7ffaac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.109+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffac008a4d0 con 0x7ffacc15b700 2024-03-04T12:12:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.110+0000 7ffaca59c640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 0x7ffaac06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0012e0 tx=0x7ffabc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.114+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffacc14c700 con 0x7ffacc15b700 2024-03-04T12:12:42.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.284+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffacc066d50 con 0x7ffacc15b700 2024-03-04T12:12:42.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.284+0000 7ffabaffd640 1 -- 172.21.15.148:0/561488521 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffac0058fe0 con 0x7ffacc15b700 2024-03-04T12:12:42.286 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:42.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 msgr2=0x7ffaac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 0x7ffaac06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0012e0 tx=0x7ffabc009040 comp rx=0 tx=0).stop 2024-03-04T12:12:42.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 msgr2=0x7ffacc0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ebe0 tx=0x7ffac000c520 comp rx=0 tx=0).stop 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 shutdown_connections 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ffaac06b4d0 0x7ffaac06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc15e100 0x7ffacc0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffacc15b700 0x7ffacc0cb6b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 --2- 172.21.15.148:0/561488521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc06bce0 0x7ffacc14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 >> 172.21.15.148:0/561488521 conn(0x7ffacc071bc0 msgr2=0x7ffacc0af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 shutdown_connections 2024-03-04T12:12:42.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.286+0000 7ffad0e59640 1 -- 172.21.15.148:0/561488521 wait complete. 2024-03-04T12:12:42.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.400+0000 7f0e8510a640 1 Processor -- start 2024-03-04T12:12:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.400+0000 7f0e8510a640 1 -- start start 2024-03-04T12:12:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e8510a640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0e800ceec0 0x7f0e800b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e8510a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e8510a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8015c790 con 0x7f0e800ccbd0 2024-03-04T12:12:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e8510a640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8015d2f0 con 0x7f0e800ceec0 2024-03-04T12:12:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e8510a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8015d6d0 con 0x7f0e8015c2e0 2024-03-04T12:12:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e7f59e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e7e59c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0e800ceec0 0x7f0e800b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.401+0000 7f0e7ed9d640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f0e800ccbd0 0x7f0e800ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:38558/0 (socket says 172.21.15.148:38558) 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7ed9d640 1 -- 172.21.15.148:0/852061518 learned_addr learned my addr 172.21.15.148:0/852061518 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7f59e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:52602/0 (socket says 172.21.15.148:52602) 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7e59c640 1 -- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 msgr2=0x7f0e800b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7e59c640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7e59c640 1 -- 172.21.15.148:0/852061518 >> v1:172.21.15.191:6789/0 conn(0x7f0e800ccbd0 legacy=0x7f0e800ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7e59c640 1 -- 172.21.15.148:0/852061518 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8015ca50 con 0x7f0e800ceec0 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.402+0000 7f0e7f59e640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e7e59c640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ceec0 0x7f0e800b7320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f164cd8a5e0b13 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e7dd9b640 1 -- 172.21.15.148:0/852061518 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68035020 con 0x7f0e800ceec0 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e7dd9b640 1 -- 172.21.15.148:0/852061518 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68031810 con 0x7f0e800ceec0 2024-03-04T12:12:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e7dd9b640 1 -- 172.21.15.148:0/852061518 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68031be0 con 0x7f0e800ceec0 2024-03-04T12:12:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 -- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ceec0 msgr2=0x7f0e800b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ceec0 0x7f0e800b7320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 -- 172.21.15.148:0/852061518 shutdown_connections 2024-03-04T12:12:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e800b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 --2- 172.21.15.148:0/852061518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ceec0 0x7f0e800b7320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 -- 172.21.15.148:0/852061518 >> 172.21.15.148:0/852061518 conn(0x7f0e80066dc0 msgr2=0x7f0e800671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.403+0000 7f0e8510a640 1 -- 172.21.15.148:0/852061518 shutdown_connections 2024-03-04T12:12:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.404+0000 7f0e8510a640 1 -- 172.21.15.148:0/852061518 wait complete. 2024-03-04T12:12:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.405+0000 7f0e8510a640 1 Processor -- start 2024-03-04T12:12:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.405+0000 7f0e8510a640 1 -- start start 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ccbd0 0x7f0e800cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e80156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ccfb0 con 0x7f0e800ceec0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8015d010 con 0x7f0e800ccbd0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e8510a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b0aa0 con 0x7f0e8015c2e0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59372/0 (socket says 172.21.15.148:59372) 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 -- 172.21.15.148:0/2180790950 learned_addr learned my addr 172.21.15.148:0/2180790950 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7f59e640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e80156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7ed9d640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ccbd0 0x7f0e800cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 -- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 msgr2=0x7f0e80156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e80156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 -- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ccbd0 msgr2=0x7f0e800cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ccbd0 0x7f0e800cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 -- 172.21.15.148:0/2180790950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800ca250 con 0x7f0e800ceec0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e7e59c640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e6802bec0 tx=0x7f0e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68041070 con 0x7f0e800ceec0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.406+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6803ad00 con 0x7f0e800ceec0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.407+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68035040 con 0x7f0e800ceec0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.407+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800ca530 con 0x7f0e800ceec0 2024-03-04T12:12:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.407+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e800caaf0 con 0x7f0e800ceec0 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.408+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e80074f60 con 0x7f0e800ceec0 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.408+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e68038020 con 0x7f0e800ceec0 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.408+0000 7f0e777fe640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.408+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0e680ac0f0 con 0x7f0e800ceec0 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.408+0000 7f0e7ed9d640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.409+0000 7f0e7ed9d640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 0x7f0e6006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0e800c60e0 tx=0x7f0e78001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.412+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e6807a4a0 con 0x7f0e800ceec0 2024-03-04T12:12:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.582+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e800b5560 con 0x7f0e800ceec0 2024-03-04T12:12:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.583+0000 7f0e777fe640 1 -- 172.21.15.148:0/2180790950 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e6807e540 con 0x7f0e800ceec0 2024-03-04T12:12:42.585 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 msgr2=0x7f0e6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 0x7f0e6006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0e800c60e0 tx=0x7f0e78001a30 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 msgr2=0x7f0e800c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e6802bec0 tx=0x7f0e680046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 shutdown_connections 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8015c2e0 0x7f0e80156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e800ceec0 0x7f0e800c9bc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 --2- 172.21.15.148:0/2180790950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e800ccbd0 0x7f0e800cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 >> 172.21.15.148:0/2180790950 conn(0x7f0e80066dc0 msgr2=0x7f0e800d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 shutdown_connections 2024-03-04T12:12:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.585+0000 7f0e8510a640 1 -- 172.21.15.148:0/2180790950 wait complete. 2024-03-04T12:12:42.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-03-04T12:12:42.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.696+0000 7fbfe47d4640 1 Processor -- start 2024-03-04T12:12:42.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.696+0000 7fbfe47d4640 1 -- start start 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe47d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe47d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe47d4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc06fd20 con 0x7fbfdc15dea0 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe47d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc06fad0 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe47d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc070000 con 0x7fbfdc0cc8d0 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58768/0 (socket says 172.21.15.148:58768) 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 -- 172.21.15.148:0/2012045685 learned_addr learned my addr 172.21.15.148:0/2012045685 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe2d71640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 -- 172.21.15.148:0/2012045685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 msgr2=0x7fbfdc0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:42.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 -- 172.21.15.148:0/2012045685 >> v1:172.21.15.191:6789/0 conn(0x7fbfdc15dea0 legacy=0x7fbfdc06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.697+0000 7fbfe1d6f640 1 -- 172.21.15.148:0/2012045685 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0702e0 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.698+0000 7fbfe1d6f640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19dc174119b0ae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.698+0000 7fbfe156e640 1 -- 172.21.15.148:0/2012045685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc034070 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.698+0000 7fbfe156e640 1 -- 172.21.15.148:0/2012045685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc030840 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.698+0000 7fbfe156e640 1 -- 172.21.15.148:0/2012045685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc030c10 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 -- 172.21.15.148:0/2012045685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc06b3e0 msgr2=0x7fbfdc0cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009870 tx=0x7fbfcc02b980 comp rx=0 tx=0).stop 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 -- 172.21.15.148:0/2012045685 shutdown_connections 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/2012045685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0cad60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 -- 172.21.15.148:0/2012045685 >> 172.21.15.148:0/2012045685 conn(0x7fbfdc0d0710 msgr2=0x7fbfdc0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:42.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 -- 172.21.15.148:0/2012045685 shutdown_connections 2024-03-04T12:12:42.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.699+0000 7fbfe47d4640 1 -- 172.21.15.148:0/2012045685 wait complete. 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.700+0000 7fbfe47d4640 1 Processor -- start 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.700+0000 7fbfe47d4640 1 -- start start 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d1380 con 0x7fbfdc15dea0 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d1200 con 0x7fbfdc06b3e0 2024-03-04T12:12:42.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d1500 con 0x7fbfdc0cc8d0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58776/0 (socket says 172.21.15.148:58776) 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 -- 172.21.15.148:0/1233387243 learned_addr learned my addr 172.21.15.148:0/1233387243 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2570640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe1d6f640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 -- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 msgr2=0x7fbfdc0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 -- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 msgr2=0x7fbfdc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 -- 172.21.15.148:0/1233387243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0c7fd0 con 0x7fbfdc15dea0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2570640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe2d71640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400e930 tx=0x7fbfd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4017070 con 0x7fbfdc15dea0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd400c980 con 0x7fbfdc15dea0 2024-03-04T12:12:42.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400cd70 con 0x7fbfdc15dea0 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc0c82b0 con 0x7fbfdc15dea0 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.701+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc0c8870 con 0x7fbfdc15dea0 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.702+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc074f60 con 0x7fbfdc15dea0 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.703+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbfd401a070 con 0x7fbfdc15dea0 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.703+0000 7fbfd2ffd640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 0x7fbfc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:42.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.703+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbfd408a5e0 con 0x7fbfdc15dea0 2024-03-04T12:12:42.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.703+0000 7fbfe2570640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 0x7fbfc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:42.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.704+0000 7fbfe2570640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 0x7fbfc006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80012e0 tx=0x7fbfd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:42.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.707+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfd4059120 con 0x7fbfdc15dea0 2024-03-04T12:12:42.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:42.947+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7fbfdc0d0f00 con 0x7fbfdc15dea0 2024-03-04T12:12:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.045+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/1233387243 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v67) v1 ==== 147+0+0 (secure 0 0 0) 0x7fbfd4058a20 con 0x7fbfdc15dea0 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 msgr2=0x7fbfc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 0x7fbfc006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80012e0 tx=0x7fbfd8009040 comp rx=0 tx=0).stop 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 msgr2=0x7fbfdc14dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400e930 tx=0x7fbfd400edf0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 shutdown_connections 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbfc006b4d0 0x7fbfc006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc15dea0 0x7fbfdc14dd10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfdc0cc8d0 0x7fbfdc0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 --2- 172.21.15.148:0/1233387243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfdc06b3e0 0x7fbfdc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 >> 172.21.15.148:0/1233387243 conn(0x7fbfdc0d0710 msgr2=0x7fbfdc0ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 shutdown_connections 2024-03-04T12:12:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.047+0000 7fbfe47d4640 1 -- 172.21.15.148:0/1233387243 wait complete. 2024-03-04T12:12:43.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-04T12:12:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.162+0000 7fb1a60fe640 1 Processor -- start 2024-03-04T12:12:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.162+0000 7fb1a60fe640 1 -- start start 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb1a60fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1a015ee80 0x7fb1a00724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb1a60fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a00cefb0 0x7fb1a00cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb1a60fe640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a00cf8f0 con 0x7fb1a006eef0 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb1a60fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00cce60 con 0x7fb1a015ee80 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb1a60fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a006f490 con 0x7fb1a00cefb0 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1a015ee80 0x7fb1a00724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1a015ee80 0x7fb1a00724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58788/0 (socket says 172.21.15.148:58788) 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 -- 172.21.15.148:0/2708894593 learned_addr learned my addr 172.21.15.148:0/2708894593 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19ffff640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a00cefb0 0x7fb1a00cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 -- 172.21.15.148:0/2708894593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a00cefb0 msgr2=0x7fb1a00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a00cefb0 0x7fb1a00cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 -- 172.21.15.148:0/2708894593 >> v1:172.21.15.191:6789/0 conn(0x7fb1a006eef0 legacy=0x7fb1a006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.163+0000 7fb19f7fe640 1 -- 172.21.15.148:0/2708894593 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a00cd0b0 con 0x7fb1a015ee80 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.164+0000 7fb19f7fe640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a015ee80 0x7fb1a00724e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0073130 tx=0x7fb19002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e0f707277367df server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.164+0000 7fb19effd640 1 -- 172.21.15.148:0/2708894593 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190035020 con 0x7fb1a015ee80 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.164+0000 7fb19effd640 1 -- 172.21.15.148:0/2708894593 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190031830 con 0x7fb1a015ee80 2024-03-04T12:12:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.164+0000 7fb19effd640 1 -- 172.21.15.148:0/2708894593 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190031c00 con 0x7fb1a015ee80 2024-03-04T12:12:43.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 -- 172.21.15.148:0/2708894593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a015ee80 msgr2=0x7fb1a00724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a015ee80 0x7fb1a00724e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0073130 tx=0x7fb19002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 -- 172.21.15.148:0/2708894593 shutdown_connections 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a00cefb0 0x7fb1a00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/2708894593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a015ee80 0x7fb1a00724e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 -- 172.21.15.148:0/2708894593 >> 172.21.15.148:0/2708894593 conn(0x7fb1a00cfad0 msgr2=0x7fb1a00cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 -- 172.21.15.148:0/2708894593 shutdown_connections 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.165+0000 7fb1a60fe640 1 -- 172.21.15.148:0/2708894593 wait complete. 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.166+0000 7fb1a60fe640 1 Processor -- start 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.166+0000 7fb1a60fe640 1 -- start start 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 0x7fb1a01e4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a015ee80 0x7fb1a01e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0072a20 con 0x7fb1a006eef0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00cd670 con 0x7fb1a00cefb0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb1a60fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a006df10 con 0x7fb1a015ee80 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58802/0 (socket says 172.21.15.148:58802) 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 -- 172.21.15.148:0/1532717307 learned_addr learned my addr 172.21.15.148:0/1532717307 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19ffff640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a015ee80 0x7fb1a01e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb196ffd640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 0x7fb1a01e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 -- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a015ee80 msgr2=0x7fb1a01e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a015ee80 0x7fb1a01e46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 -- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 msgr2=0x7fb1a01e4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 0x7fb1a01e4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 -- 172.21.15.148:0/1532717307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a00cc710 con 0x7fb1a006eef0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb196ffd640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 0x7fb1a01e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.167+0000 7fb19f7fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb190000c00 tx=0x7fb1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.168+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190040070 con 0x7fb1a006eef0 2024-03-04T12:12:43.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.168+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190039940 con 0x7fb1a006eef0 2024-03-04T12:12:43.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.168+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190035040 con 0x7fb1a006eef0 2024-03-04T12:12:43.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.168+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a014ec00 con 0x7fb1a006eef0 2024-03-04T12:12:43.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.168+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a013c100 con 0x7fb1a006eef0 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.169+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a00cd9c0 con 0x7fb1a006eef0 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.169+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb19004b060 con 0x7fb1a006eef0 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.170+0000 7fb19cff9640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.170+0000 7fb196ffd640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.170+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb1900abe70 con 0x7fb1a006eef0 2024-03-04T12:12:43.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.170+0000 7fb196ffd640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 0x7fb18006d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0155530 tx=0x7fb188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.174+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1a00cd9c0 con 0x7fb1a006eef0 2024-03-04T12:12:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.345+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1a0074f60 con 0x7fb1a006eef0 2024-03-04T12:12:43.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.348+0000 7fb19cff9640 1 -- 172.21.15.148:0/1532717307 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1a00cf8f0 con 0x7fb1a006eef0 2024-03-04T12:12:43.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 msgr2=0x7fb18006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 0x7fb18006d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0155530 tx=0x7fb188009040 comp rx=0 tx=0).stop 2024-03-04T12:12:43.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 msgr2=0x7fb1a00cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb190000c00 tx=0x7fb1900046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 shutdown_connections 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb18006b4d0 0x7fb18006d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1a015ee80 0x7fb1a01e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00cefb0 0x7fb1a01e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 --2- 172.21.15.148:0/1532717307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1a006eef0 0x7fb1a00cc1d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.350+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 >> 172.21.15.148:0/1532717307 conn(0x7fb1a00cfad0 msgr2=0x7fb1a015add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.351+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 shutdown_connections 2024-03-04T12:12:43.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.351+0000 7fb1a60fe640 1 -- 172.21.15.148:0/1532717307 wait complete. 2024-03-04T12:12:43.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-04T12:12:43.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-04T12:12:43.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-04T12:12:43.449 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-04T12:12:43.449 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:43.450 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:43.465 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:43.465 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:43.520 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:43.520 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:43.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:43.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.713+0000 7f0076420640 1 Processor -- start 2024-03-04T12:12:43.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.713+0000 7f0076420640 1 -- start start 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f0076420640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f007015de00 0x7f00700b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f0076420640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007015e1e0 con 0x7f00700b5470 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f0076420640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00700d1c10 con 0x7f00700b9380 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f0076420640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00700d2820 con 0x7f007015de00 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f00749bd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f00700b5470 0x7f00700d2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50138/0 (socket says 172.21.15.148:50138) 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.714+0000 7f00749bd640 1 -- 172.21.15.148:0/3691274902 learned_addr learned my addr 172.21.15.148:0/3691274902 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784574969 0 0) 0x7f007015e1e0 con 0x7f00700b5470 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0058003520 con 0x7f00700b5470 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006ffff640 1 --2- 172.21.15.148:0/3691274902 >> v2:172.21.15.191:3300/0 conn(0x7f007015de00 0x7f00700b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 830395795 0 0) 0x7f0058003520 con 0x7f00700b5470 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 >> v2:172.21.15.191:3300/0 conn(0x7f007015de00 msgr2=0x7f00700b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 --2- 172.21.15.148:0/3691274902 >> v2:172.21.15.191:3300/0 conn(0x7f007015de00 0x7f00700b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 >> v1:172.21.15.191:6789/0 conn(0x7f00700b9380 legacy=0x7f00700b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700d1ef0 con 0x7f00700b5470 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f00680019f0 con 0x7f00700b5470 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 >> v1:172.21.15.148:6789/0 conn(0x7f00700b5470 legacy=0x7f00700d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580039b0 0x7f0058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0058004800 0x7f0058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 0x7f005800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700d2820 con 0x7f00580051c0 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005800c060 con 0x7f0058004800 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005800c260 con 0x7f00580039b0 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 0x7f005800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006f7fe640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0058004800 0x7f0058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f006ffff640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580039b0 0x7f0058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 -- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580039b0 msgr2=0x7f0058003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580039b0 0x7f0058003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 -- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0058004800 msgr2=0x7f0058004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0058004800 0x7f0058004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.715+0000 7f00749bd640 1 -- 172.21.15.148:0/3691274902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058003520 con 0x7f00580051c0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.716+0000 7f00749bd640 1 -- 172.21.15.148:0/3691274902 reap_dead start 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.716+0000 7f00749bd640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 0x7f005800b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f006800a840 tx=0x7f0058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42a8c7c9a33bb9b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.716+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800a460 con 0x7f00580051c0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.716+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068034670 con 0x7f00580051c0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.716+0000 7f006effd640 1 -- 172.21.15.148:0/3691274902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068034a60 con 0x7f00580051c0 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 -- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 msgr2=0x7f005800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 0x7f005800b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f006800a840 tx=0x7f0058005830 comp rx=0 tx=0).stop 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 -- 172.21.15.148:0/3691274902 shutdown_connections 2024-03-04T12:12:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 --2- 172.21.15.148:0/3691274902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00580051c0 0x7f005800b9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 -- 172.21.15.148:0/3691274902 >> 172.21.15.148:0/3691274902 conn(0x7f00700d0470 msgr2=0x7f00700d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 -- 172.21.15.148:0/3691274902 shutdown_connections 2024-03-04T12:12:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.717+0000 7f0076420640 1 -- 172.21.15.148:0/3691274902 wait complete. 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.718+0000 7f0076420640 1 Processor -- start 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.718+0000 7f0076420640 1 -- start start 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580051c0 0x7f0070159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0070157000 0x7f0070160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700b97e0 con 0x7f0070145dc0 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700d24b0 con 0x7f0070157000 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f0076420640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00700b2e60 con 0x7f00580051c0 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58830/0 (socket says 172.21.15.148:58830) 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 -- 172.21.15.148:0/3321007337 learned_addr learned my addr 172.21.15.148:0/3321007337 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f006ffff640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580051c0 0x7f0070159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f006f7fe640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0070157000 0x7f0070160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 -- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580051c0 msgr2=0x7f0070159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580051c0 0x7f0070159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 -- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0070157000 msgr2=0x7f0070160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0070157000 0x7f0070160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 -- 172.21.15.148:0/3321007337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070146780 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.719+0000 7f00749bd640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f006802cc50 tx=0x7f00700d1de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.720+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068039040 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.720+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068043070 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.720+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068001fb0 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.720+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0070157480 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.720+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070155740 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.721+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0070074f60 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.721+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f00680019b0 con 0x7f0070145dc0 2024-03-04T12:12:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.721+0000 7f006cff9640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.722+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f00680ac200 con 0x7f0070145dc0 2024-03-04T12:12:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.722+0000 7f006ffff640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.722+0000 7f006ffff640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 0x7f005c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0060009b40 tx=0x7f0060009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.726+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f006807a7e0 con 0x7f0070145dc0 2024-03-04T12:12:43.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.899+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00700d0850 con 0x7f0070145dc0 2024-03-04T12:12:43.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.900+0000 7f006cff9640 1 -- 172.21.15.148:0/3321007337 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f00680ac020 con 0x7f0070145dc0 2024-03-04T12:12:43.902 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:43.902 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:43.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 msgr2=0x7f005c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 0x7f005c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0060009b40 tx=0x7f0060009190 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 msgr2=0x7f0070146240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f006802cc50 tx=0x7f00700d1de0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 shutdown_connections 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070145dc0 0x7f0070146240 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0070157000 0x7f0070160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 --2- 172.21.15.148:0/3321007337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00580051c0 0x7f0070159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.902+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 >> 172.21.15.148:0/3321007337 conn(0x7f00700d0470 msgr2=0x7f00700b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.903+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 shutdown_connections 2024-03-04T12:12:43.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:43.903+0000 7f0076420640 1 -- 172.21.15.148:0/3321007337 wait complete. 2024-03-04T12:12:43.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:43.916 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.017+0000 7fe5ce452640 1 Processor -- start 2024-03-04T12:12:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 -- start start 2024-03-04T12:12:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5c806c060 0x7fe5c80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe5c80cec20 0x7fe5c80cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c80cf740 con 0x7fe5c8161390 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806d3e0 con 0x7fe5c806c060 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5ce452640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806c7d0 con 0x7fe5c80cec20 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5c77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5c806c060 0x7fe5c80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5c7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe5c8161390 0x7fe5c80d20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50154/0 (socket says 172.21.15.148:50154) 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.018+0000 7fe5c7fff640 1 -- 172.21.15.148:0/3411870964 learned_addr learned my addr 172.21.15.148:0/3411870964 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c77fe640 1 -- 172.21.15.148:0/3411870964 >> v2:172.21.15.191:3300/0 conn(0x7fe5c80cec20 msgr2=0x7fe5c80cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c77fe640 1 --2- 172.21.15.148:0/3411870964 >> v2:172.21.15.191:3300/0 conn(0x7fe5c80cec20 0x7fe5c80cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c77fe640 1 -- 172.21.15.148:0/3411870964 >> v1:172.21.15.148:6789/0 conn(0x7fe5c8161390 legacy=0x7fe5c80d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c77fe640 1 -- 172.21.15.148:0/3411870964 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c806cab0 con 0x7fe5c806c060 2024-03-04T12:12:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c6ffd640 1 -- 172.21.15.148:0/3411870964 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983732414 0 0) 0x7fe5c80cf740 con 0x7fe5c8161390 2024-03-04T12:12:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.019+0000 7fe5c77fe640 1 --2- 172.21.15.148:0/3411870964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c806c060 0x7fe5c80ce6e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80ce270 tx=0x7fe5bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c219d1cd51d9fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5c6ffd640 1 -- 172.21.15.148:0/3411870964 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc035020 con 0x7fe5c806c060 2024-03-04T12:12:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5c6ffd640 1 -- 172.21.15.148:0/3411870964 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc031830 con 0x7fe5c806c060 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5c6ffd640 1 -- 172.21.15.148:0/3411870964 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc031c00 con 0x7fe5c806c060 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 -- 172.21.15.148:0/3411870964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c806c060 msgr2=0x7fe5c80ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 --2- 172.21.15.148:0/3411870964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c806c060 0x7fe5c80ce6e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80ce270 tx=0x7fe5bc02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 -- 172.21.15.148:0/3411870964 shutdown_connections 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 --2- 172.21.15.148:0/3411870964 >> v2:172.21.15.191:3300/0 conn(0x7fe5c80cec20 0x7fe5c80cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 --2- 172.21.15.148:0/3411870964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c806c060 0x7fe5c80ce6e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 -- 172.21.15.148:0/3411870964 >> 172.21.15.148:0/3411870964 conn(0x7fe5c8066dc0 msgr2=0x7fe5c80671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 -- 172.21.15.148:0/3411870964 shutdown_connections 2024-03-04T12:12:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.020+0000 7fe5ce452640 1 -- 172.21.15.148:0/3411870964 wait complete. 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.021+0000 7fe5ce452640 1 Processor -- start 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 -- start start 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c806c060 0x7fe5c8159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c8161390 0x7fe5c814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806c4c0 con 0x7fe5c80cec20 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806d070 con 0x7fe5c806c060 2024-03-04T12:12:44.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5ce452640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8161770 con 0x7fe5c8161390 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58846/0 (socket says 172.21.15.148:58846) 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.022+0000 7fe5c77fe640 1 -- 172.21.15.148:0/2139443551 learned_addr learned my addr 172.21.15.148:0/2139443551 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 -- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c8161390 msgr2=0x7fe5c814fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c8161390 0x7fe5c814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 -- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c806c060 msgr2=0x7fe5c8159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c806c060 0x7fe5c8159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 -- 172.21.15.148:0/2139443551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c80b4500 con 0x7fe5c80cec20 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c77fe640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009890 tx=0x7fe5bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc03ba80 con 0x7fe5c80cec20 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc03bc40 con 0x7fe5c80cec20 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c80b47e0 con 0x7fe5c80cec20 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0453f0 con 0x7fe5c80cec20 2024-03-04T12:12:44.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.023+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c80c9de0 con 0x7fe5c80cec20 2024-03-04T12:12:44.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.024+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8074f60 con 0x7fe5c80cec20 2024-03-04T12:12:44.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.025+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe5bc038020 con 0x7fe5c80cec20 2024-03-04T12:12:44.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.025+0000 7fe5c4ff9640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 0x7fe5b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.025+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe5bc0ac3f0 con 0x7fe5c80cec20 2024-03-04T12:12:44.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.025+0000 7fe5c7fff640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 0x7fe5b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.025+0000 7fe5c7fff640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 0x7fe5b406d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007fd0 tx=0x7fe5b8007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.029+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5bc040070 con 0x7fe5c80cec20 2024-03-04T12:12:44.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.200+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5c80cd440 con 0x7fe5c80cec20 2024-03-04T12:12:44.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.201+0000 7fe5c4ff9640 1 -- 172.21.15.148:0/2139443551 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5bc07ae60 con 0x7fe5c80cec20 2024-03-04T12:12:44.203 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:44.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 msgr2=0x7fe5b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 0x7fe5b406d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007fd0 tx=0x7fe5b8007960 comp rx=0 tx=0).stop 2024-03-04T12:12:44.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 msgr2=0x7fe5c80b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009890 tx=0x7fe5bc0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 shutdown_connections 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe5b406b4d0 0x7fe5b406d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c8161390 0x7fe5c814fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c80cec20 0x7fe5c80b3d90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 --2- 172.21.15.148:0/2139443551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c806c060 0x7fe5c8159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 >> 172.21.15.148:0/2139443551 conn(0x7fe5c8066dc0 msgr2=0x7fe5c806fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 shutdown_connections 2024-03-04T12:12:44.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.203+0000 7fe5ce452640 1 -- 172.21.15.148:0/2139443551 wait complete. 2024-03-04T12:12:44.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:44.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.316+0000 7fa529bcd640 1 Processor -- start 2024-03-04T12:12:44.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.316+0000 7fa529bcd640 1 -- start start 2024-03-04T12:12:44.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa529bcd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa52415eb70 0x7fa5240b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa529bcd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52415ca50 con 0x7fa5240ccbd0 2024-03-04T12:12:44.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa529bcd640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52415cd30 con 0x7fa5240ceec0 2024-03-04T12:12:44.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa529bcd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52415d2f0 con 0x7fa52415eb70 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa523fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa52415eb70 0x7fa5240b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa5237fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa5240ccbd0 0x7fa5240ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50158/0 (socket says 172.21.15.148:50158) 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.317+0000 7fa5237fe640 1 -- 172.21.15.148:0/3437792816 learned_addr learned my addr 172.21.15.148:0/3437792816 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa522ffd640 1 -- 172.21.15.148:0/3437792816 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265929147 0 0) 0x7fa52415ca50 con 0x7fa5240ccbd0 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa522ffd640 1 -- 172.21.15.148:0/3437792816 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa510003520 con 0x7fa5240ccbd0 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa523fff640 1 -- 172.21.15.148:0/3437792816 >> v1:172.21.15.191:6789/0 conn(0x7fa5240ceec0 legacy=0x7fa5240b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa523fff640 1 -- 172.21.15.148:0/3437792816 >> v1:172.21.15.148:6789/0 conn(0x7fa5240ccbd0 legacy=0x7fa5240ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa523fff640 1 -- 172.21.15.148:0/3437792816 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52415d010 con 0x7fa52415eb70 2024-03-04T12:12:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa523fff640 1 --2- 172.21.15.148:0/3437792816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52415eb70 0x7fa5240b0c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009d90 tx=0x7fa51c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3e45ead1caea5c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.318+0000 7fa522ffd640 1 -- 172.21.15.148:0/3437792816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c039070 con 0x7fa52415eb70 2024-03-04T12:12:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa522ffd640 1 -- 172.21.15.148:0/3437792816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c02cc60 con 0x7fa52415eb70 2024-03-04T12:12:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa522ffd640 1 -- 172.21.15.148:0/3437792816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c034640 con 0x7fa52415eb70 2024-03-04T12:12:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 -- 172.21.15.148:0/3437792816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52415eb70 msgr2=0x7fa5240b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 --2- 172.21.15.148:0/3437792816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52415eb70 0x7fa5240b0c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009d90 tx=0x7fa51c02beb0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 -- 172.21.15.148:0/3437792816 shutdown_connections 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 --2- 172.21.15.148:0/3437792816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52415eb70 0x7fa5240b0c70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 -- 172.21.15.148:0/3437792816 >> 172.21.15.148:0/3437792816 conn(0x7fa524066dc0 msgr2=0x7fa5240671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 -- 172.21.15.148:0/3437792816 shutdown_connections 2024-03-04T12:12:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.319+0000 7fa529bcd640 1 -- 172.21.15.148:0/3437792816 wait complete. 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.320+0000 7fa529bcd640 1 Processor -- start 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.320+0000 7fa529bcd640 1 -- start start 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5240ccbd0 0x7fa524152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa52415eb70 0x7fa5240d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240b7320 con 0x7fa5240ceec0 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52415d6d0 con 0x7fa52415eb70 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa529bcd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240b74a0 con 0x7fa5240ccbd0 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa51affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa51affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58854/0 (socket says 172.21.15.148:58854) 2024-03-04T12:12:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.321+0000 7fa51affd640 1 -- 172.21.15.148:0/228996097 learned_addr learned my addr 172.21.15.148:0/228996097 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 -- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5240ccbd0 msgr2=0x7fa524152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5240ccbd0 0x7fa524152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa523fff640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa52415eb70 0x7fa5240d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 -- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa52415eb70 msgr2=0x7fa5240d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa52415eb70 0x7fa5240d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 -- 172.21.15.148:0/228996097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240c8e10 con 0x7fa5240ceec0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa51affd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c460 tx=0x7fa50c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c016020 con 0x7fa5240ceec0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c012870 con 0x7fa5240ceec0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5240c90f0 con 0x7fa5240ceec0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c012c60 con 0x7fa5240ceec0 2024-03-04T12:12:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.322+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5240c96b0 con 0x7fa5240ceec0 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.323+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524074f60 con 0x7fa5240ceec0 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.323+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa50c019020 con 0x7fa5240ceec0 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.324+0000 7fa520ff9640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 0x7fa50406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.324+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa50c08a2b0 con 0x7fa5240ceec0 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.324+0000 7fa5237fe640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 0x7fa50406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.324+0000 7fa5237fe640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 0x7fa50406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa514000f60 tx=0x7fa5140078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.328+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa50c020030 con 0x7fa5240ceec0 2024-03-04T12:12:44.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.499+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa524156240 con 0x7fa5240ceec0 2024-03-04T12:12:44.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.499+0000 7fa520ff9640 1 -- 172.21.15.148:0/228996097 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa50c019310 con 0x7fa5240ceec0 2024-03-04T12:12:44.501 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:44.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 msgr2=0x7fa50406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 0x7fa50406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa514000f60 tx=0x7fa5140078f0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 msgr2=0x7fa524149a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c460 tx=0x7fa50c00c920 comp rx=0 tx=0).stop 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 shutdown_connections 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa50406b4d0 0x7fa50406d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa52415eb70 0x7fa5240d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5240ceec0 0x7fa524149a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.501+0000 7fa529bcd640 1 --2- 172.21.15.148:0/228996097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5240ccbd0 0x7fa524152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.502+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 >> 172.21.15.148:0/228996097 conn(0x7fa524066dc0 msgr2=0x7fa5240cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.502+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 shutdown_connections 2024-03-04T12:12:44.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.502+0000 7fa529bcd640 1 -- 172.21.15.148:0/228996097 wait complete. 2024-03-04T12:12:44.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.618+0000 7fb95e4c0640 1 Processor -- start 2024-03-04T12:12:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.618+0000 7fb95e4c0640 1 -- start start 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb95e4c0640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb95815a810 0x7fb9580b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb95e4c0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95806d8a0 con 0x7fb95806d3c0 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb95e4c0640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95806db80 con 0x7fb9580cbfb0 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb95e4c0640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95806de60 con 0x7fb95815a810 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb9577fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb95806d3c0 0x7fb95815abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50160/0 (socket says 172.21.15.148:50160) 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb9577fe640 1 -- 172.21.15.148:0/611678438 learned_addr learned my addr 172.21.15.148:0/611678438 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704356723 0 0) 0x7fb95806d8a0 con 0x7fb95806d3c0 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb940005730 con 0x7fb95806d3c0 2024-03-04T12:12:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb957fff640 1 --2- 172.21.15.148:0/611678438 >> v2:172.21.15.191:3300/0 conn(0x7fb95815a810 0x7fb9580b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474852661 0 0) 0x7fb940005730 con 0x7fb95806d3c0 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 >> v2:172.21.15.191:3300/0 conn(0x7fb95815a810 msgr2=0x7fb9580b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 --2- 172.21.15.148:0/611678438 >> v2:172.21.15.191:3300/0 conn(0x7fb95815a810 0x7fb9580b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 >> v1:172.21.15.191:6789/0 conn(0x7fb9580cbfb0 legacy=0x7fb95815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95806e140 con 0x7fb95806d3c0 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fb94c001670 con 0x7fb95806d3c0 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 >> v1:172.21.15.148:6789/0 conn(0x7fb95806d3c0 legacy=0x7fb95815abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb940005bc0 0x7fb940005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb940006a10 0x7fb940006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb94000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95806de60 con 0x7fb9400073d0 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94000e270 con 0x7fb940006a10 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.619+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94000e470 con 0x7fb940005bc0 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb957fff640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb940005bc0 0x7fb940005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb94000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb9577fe640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb940006a10 0x7fb940006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 -- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb940005bc0 msgr2=0x7fb940005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb940005bc0 0x7fb940005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 -- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb940006a10 msgr2=0x7fb940006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb940006a10 0x7fb940006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.620+0000 7fb95ca5d640 1 -- 172.21.15.148:0/611678438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940005730 con 0x7fb9400073d0 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95ca5d640 1 -- 172.21.15.148:0/611678438 reap_dead start 2024-03-04T12:12:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95ca5d640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb94000dbd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb950001e50 tx=0x7fb940007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb478cce9fc022aa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950001a70 con 0x7fb9400073d0 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb950003bb0 con 0x7fb9400073d0 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb956ffd640 1 -- 172.21.15.148:0/611678438 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9500050d0 con 0x7fb9400073d0 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 -- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 msgr2=0x7fb94000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb94000dbd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb950001e50 tx=0x7fb940007a40 comp rx=0 tx=0).stop 2024-03-04T12:12:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 -- 172.21.15.148:0/611678438 shutdown_connections 2024-03-04T12:12:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/611678438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb94000dbd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 -- 172.21.15.148:0/611678438 >> 172.21.15.148:0/611678438 conn(0x7fb95806acd0 msgr2=0x7fb95806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 -- 172.21.15.148:0/611678438 shutdown_connections 2024-03-04T12:12:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.621+0000 7fb95e4c0640 1 -- 172.21.15.148:0/611678438 wait complete. 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.622+0000 7fb95e4c0640 1 Processor -- start 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 -- start start 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958144b90 0x7fb9580a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb95815cff0 0x7fb958143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580b4190 con 0x7fb9400073d0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815e5d0 con 0x7fb958144b90 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb95e4c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580b4310 con 0x7fb95815cff0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb957fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb957fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58866/0 (socket says 172.21.15.148:58866) 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.623+0000 7fb957fff640 1 -- 172.21.15.148:0/1387854582 learned_addr learned my addr 172.21.15.148:0/1387854582 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb9577fe640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958144b90 0x7fb9580a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb95ca5d640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb95815cff0 0x7fb958143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 -- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb95815cff0 msgr2=0x7fb958143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb95815cff0 0x7fb958143f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 -- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958144b90 msgr2=0x7fb9580a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958144b90 0x7fb9580a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 -- 172.21.15.148:0/1387854582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580a89a0 con 0x7fb9400073d0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb957fff640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb948011910 tx=0x7fb948011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94800faf0 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94800fc90 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948014440 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb958141ef0 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.624+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9581424b0 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.625+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958074f60 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.626+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb948012040 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.626+0000 7fb954ff9640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 0x7fb94406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.626+0000 7fb9577fe640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 0x7fb94406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.626+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb94808aaf0 con 0x7fb9400073d0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.627+0000 7fb9577fe640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 0x7fb94406d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb958156de0 tx=0x7fb94c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.630+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb948019030 con 0x7fb9400073d0 2024-03-04T12:12:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.801+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9580722b0 con 0x7fb9400073d0 2024-03-04T12:12:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.802+0000 7fb954ff9640 1 -- 172.21.15.148:0/1387854582 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb948059600 con 0x7fb9400073d0 2024-03-04T12:12:44.804 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 msgr2=0x7fb94406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 0x7fb94406d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb958156de0 tx=0x7fb94c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 msgr2=0x7fb958156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb948011910 tx=0x7fb948011dd0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 shutdown_connections 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fb94406b500 0x7fb94406d9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb95815cff0 0x7fb958143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958144b90 0x7fb9580a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 --2- 172.21.15.148:0/1387854582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9400073d0 0x7fb958156e50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 >> 172.21.15.148:0/1387854582 conn(0x7fb95806acd0 msgr2=0x7fb958154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 shutdown_connections 2024-03-04T12:12:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.804+0000 7fb95e4c0640 1 -- 172.21.15.148:0/1387854582 wait complete. 2024-03-04T12:12:44.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:44.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.918+0000 7ff00e5d4640 1 Processor -- start 2024-03-04T12:12:44.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.918+0000 7ff00e5d4640 1 -- start start 2024-03-04T12:12:44.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00e5d4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff0080cf660 0x7ff0080cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00e5d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7ff008161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00e5d4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff008162450 con 0x7ff00815da20 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00e5d4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080d0000 con 0x7ff0080cf660 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00e5d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cc920 con 0x7ff0081616e0 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff007fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff00815da20 0x7ff0080ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50176/0 (socket says 172.21.15.148:50176) 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff007fff640 1 -- 172.21.15.148:0/815209816 learned_addr learned my addr 172.21.15.148:0/815209816 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff00cb71640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7ff008161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff0077fe640 1 --2- 172.21.15.148:0/815209816 >> v2:172.21.15.191:3300/0 conn(0x7ff0080cf660 0x7ff0080cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.919+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039183300 0 0) 0x7ff008162450 con 0x7ff00815da20 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff4003520 con 0x7ff00815da20 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122214045 0 0) 0x7feff4003520 con 0x7ff00815da20 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 msgr2=0x7ff008161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7ff008161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 >> v2:172.21.15.191:3300/0 conn(0x7ff0080cf660 msgr2=0x7ff0080cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 --2- 172.21.15.148:0/815209816 >> v2:172.21.15.191:3300/0 conn(0x7ff0080cf660 0x7ff0080cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080cc2d0 con 0x7ff00815da20 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7feff8001670 con 0x7ff00815da20 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 >> v1:172.21.15.148:6789/0 conn(0x7ff00815da20 legacy=0x7ff0080ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40039b0 0x7feff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7feff4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff4004d40 0x7feff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cc920 con 0x7feff40039b0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff400b9c0 con 0x7feff4004d40 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff400bb70 con 0x7ff0081616e0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40039b0 0x7feff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff0077fe640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7feff4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff00cb71640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff4004d40 0x7feff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 -- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 msgr2=0x7feff4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7feff4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 -- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff4004d40 msgr2=0x7feff40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff4004d40 0x7feff40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff007fff640 1 -- 172.21.15.148:0/815209816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4003520 con 0x7feff40039b0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.920+0000 7ff0077fe640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0081616e0 0x7feff4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.921+0000 7ff007fff640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40039b0 0x7feff4003dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feff8001050 tx=0x7feff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fcecc1e6457a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.921+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8007db0 con 0x7feff40039b0 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.921+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8030070 con 0x7feff40039b0 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.921+0000 7ff006ffd640 1 -- 172.21.15.148:0/815209816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff802cdc0 con 0x7feff40039b0 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 -- 172.21.15.148:0/815209816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40039b0 msgr2=0x7feff4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/815209816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40039b0 0x7feff4003dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feff8001050 tx=0x7feff4005830 comp rx=0 tx=0).stop 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00cb71640 1 -- 172.21.15.148:0/815209816 reap_dead start 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 -- 172.21.15.148:0/815209816 shutdown_connections 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 -- 172.21.15.148:0/815209816 >> 172.21.15.148:0/815209816 conn(0x7ff0080b33a0 msgr2=0x7ff0080b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 -- 172.21.15.148:0/815209816 shutdown_connections 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.922+0000 7ff00e5d4640 1 -- 172.21.15.148:0/815209816 wait complete. 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.923+0000 7ff00e5d4640 1 Processor -- start 2024-03-04T12:12:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.923+0000 7ff00e5d4640 1 -- start start 2024-03-04T12:12:44.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff40039b0 0x7ff008160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff008157a50 0x7ff0080ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00815de00 con 0x7ff008150950 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cbf80 con 0x7ff008157a50 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff008161b70 con 0x7feff40039b0 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58872/0 (socket says 172.21.15.148:58872) 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 -- 172.21.15.148:0/2880106119 learned_addr learned my addr 172.21.15.148:0/2880106119 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff007fff640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff40039b0 0x7ff008160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00cb71640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff008157a50 0x7ff0080ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 -- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff40039b0 msgr2=0x7ff008160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff40039b0 0x7ff008160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 -- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff008157a50 msgr2=0x7ff0080ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff008157a50 0x7ff0080ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 -- 172.21.15.148:0/2880106119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008156470 con 0x7ff008150950 2024-03-04T12:12:44.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff0077fe640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feff0011600 tx=0x7feff0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000f920 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000fac0 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff008156750 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000fe90 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.924+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff008156d10 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.925+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff008074f60 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.926+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feff0015460 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.926+0000 7ff004ff9640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 0x7fefe806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.926+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feff008ae20 con 0x7ff008150950 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.926+0000 7ff007fff640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 0x7fefe806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:44.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.927+0000 7ff007fff640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 0x7fefe806db60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feff8034000 tx=0x7feff8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:44.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:44.930+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff0019030 con 0x7ff008150950 2024-03-04T12:12:45.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.107+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff0080d0220 con 0x7ff008150950 2024-03-04T12:12:45.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.108+0000 7ff004ff9640 1 -- 172.21.15.148:0/2880106119 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7feff0059930 con 0x7ff008150950 2024-03-04T12:12:45.109 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:45.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 msgr2=0x7fefe806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 0x7fefe806db60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feff8034000 tx=0x7feff8007000 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 msgr2=0x7ff008157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feff0011600 tx=0x7feff0011ac0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 shutdown_connections 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fefe806b6a0 0x7fefe806db60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff008157a50 0x7ff0080ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff008150950 0x7ff008157510 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 --2- 172.21.15.148:0/2880106119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff40039b0 0x7ff008160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 >> 172.21.15.148:0/2880106119 conn(0x7ff0080b33a0 msgr2=0x7ff0080b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 shutdown_connections 2024-03-04T12:12:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.110+0000 7ff00e5d4640 1 -- 172.21.15.148:0/2880106119 wait complete. 2024-03-04T12:12:45.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:45.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.223+0000 7f5b00559640 1 Processor -- start 2024-03-04T12:12:45.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.223+0000 7f5b00559640 1 -- start start 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5b00559640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5af815de00 0x7f5af80b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5b00559640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5af80b9380 0x7f5af80b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5b00559640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af815e1e0 con 0x7f5af80b70d0 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5b00559640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80d2820 con 0x7f5af815de00 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5b00559640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80d1c10 con 0x7f5af80b9380 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.224+0000 7f5afe2f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5af815de00 0x7f5af80b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afeaf6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5af80b70d0 0x7f5af80b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50180/0 (socket says 172.21.15.148:50180) 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afeaf6640 1 -- 172.21.15.148:0/2891583483 learned_addr learned my addr 172.21.15.148:0/2891583483 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afe2f5640 1 -- 172.21.15.148:0/2891583483 >> v2:172.21.15.191:3300/0 conn(0x7f5af80b9380 msgr2=0x7f5af80b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2891583483 >> v2:172.21.15.191:3300/0 conn(0x7f5af80b9380 0x7f5af80b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afe2f5640 1 -- 172.21.15.148:0/2891583483 >> v1:172.21.15.148:6789/0 conn(0x7f5af80b70d0 legacy=0x7f5af80b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afe2f5640 1 -- 172.21.15.148:0/2891583483 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80d1ef0 con 0x7f5af815de00 2024-03-04T12:12:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.225+0000 7f5afd2f3640 1 -- 172.21.15.148:0/2891583483 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865790067 0 0) 0x7f5af815e1e0 con 0x7f5af80b70d0 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2891583483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815de00 0x7f5af80b2e60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5af8161830 tx=0x7f5ae802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71723ae63f2e4de server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5afd2f3640 1 -- 172.21.15.148:0/2891583483 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8035020 con 0x7f5af815de00 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5afd2f3640 1 -- 172.21.15.148:0/2891583483 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8031830 con 0x7f5af815de00 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5afd2f3640 1 -- 172.21.15.148:0/2891583483 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8031c00 con 0x7f5af815de00 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 -- 172.21.15.148:0/2891583483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815de00 msgr2=0x7f5af80b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 --2- 172.21.15.148:0/2891583483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815de00 0x7f5af80b2e60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5af8161830 tx=0x7f5ae802b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 -- 172.21.15.148:0/2891583483 shutdown_connections 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 --2- 172.21.15.148:0/2891583483 >> v2:172.21.15.191:3300/0 conn(0x7f5af80b9380 0x7f5af80b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 --2- 172.21.15.148:0/2891583483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815de00 0x7f5af80b2e60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 -- 172.21.15.148:0/2891583483 >> 172.21.15.148:0/2891583483 conn(0x7f5af80d0470 msgr2=0x7f5af80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 -- 172.21.15.148:0/2891583483 shutdown_connections 2024-03-04T12:12:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.226+0000 7f5b00559640 1 -- 172.21.15.148:0/2891583483 wait complete. 2024-03-04T12:12:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.227+0000 7f5b00559640 1 Processor -- start 2024-03-04T12:12:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.227+0000 7f5b00559640 1 -- start start 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 0x7f5af8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 0x7f5af815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80b97e0 con 0x7f5af80b9380 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80d24b0 con 0x7f5af80b70d0 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5b00559640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80b5b80 con 0x7f5af815de00 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5afdaf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5afdaf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58888/0 (socket says 172.21.15.148:58888) 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5afdaf4640 1 -- 172.21.15.148:0/2226541402 learned_addr learned my addr 172.21.15.148:0/2226541402 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5afeaf6640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 0x7f5af815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.228+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 0x7f5af8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 -- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 msgr2=0x7f5af815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 0x7f5af815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 -- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 msgr2=0x7f5af8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 0x7f5af8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 -- 172.21.15.148:0/2226541402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af81577b0 con 0x7f5af80b9380 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 0x7f5af8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afeaf6640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 0x7f5af815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5afdaf4640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b4d0 tx=0x7f5af400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4014070 con 0x7f5af80b9380 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af4010610 con 0x7f5af80b9380 2024-03-04T12:12:45.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af81582a0 con 0x7f5af80b9380 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4010a00 con 0x7f5af80b9380 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.229+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af8158860 con 0x7f5af80b9380 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.230+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af8074f60 con 0x7f5af80b9380 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.230+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5af4017070 con 0x7f5af80b9380 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.231+0000 7f5aeeffd640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.231+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.231+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5af4089ee0 con 0x7f5af80b9380 2024-03-04T12:12:45.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.231+0000 7f5afe2f5640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 0x7f5adc06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8036000 tx=0x7f5ae8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.235+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5af40589f0 con 0x7f5af80b9380 2024-03-04T12:12:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.420+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5af80b7fd0 con 0x7f5af80b9380 2024-03-04T12:12:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.420+0000 7f5aeeffd640 1 -- 172.21.15.148:0/2226541402 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5af40582f0 con 0x7f5af80b9380 2024-03-04T12:12:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 msgr2=0x7f5adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 0x7f5adc06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8036000 tx=0x7f5ae8008030 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 msgr2=0x7f5af81579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b4d0 tx=0x7f5af400b990 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 shutdown_connections 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5af815de00 0x7f5af815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80b9380 0x7f5af81579c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 --2- 172.21.15.148:0/2226541402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5af80b70d0 0x7f5af8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 >> 172.21.15.148:0/2226541402 conn(0x7f5af80d0470 msgr2=0x7f5af815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 shutdown_connections 2024-03-04T12:12:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.422+0000 7f5b00559640 1 -- 172.21.15.148:0/2226541402 wait complete. 2024-03-04T12:12:45.436 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T12:12:45.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:45.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-04T12:12:45.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:12:45.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:45.465 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-04T12:12:45.520 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:45.521 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:45.539 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:45.540 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:45.597 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:45.598 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:45.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.757+0000 7f1a47759640 1 Processor -- start 2024-03-04T12:12:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.757+0000 7f1a47759640 1 -- start start 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a47759640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1a40154a80 0x7f1a4015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a47759640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a401624e0 con 0x7f1a400cfae0 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a47759640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400cd010 con 0x7f1a400cf650 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a47759640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a40071df0 con 0x7f1a40154a80 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a45cf6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1a400cfae0 0x7f1a40161dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50192/0 (socket says 172.21.15.148:50192) 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a45cf6640 1 -- 172.21.15.148:0/66779662 learned_addr learned my addr 172.21.15.148:0/66779662 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a454f5640 1 --2- 172.21.15.148:0/66779662 >> v2:172.21.15.191:3300/0 conn(0x7f1a40154a80 0x7f1a4015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.758+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284211478 0 0) 0x7f1a401624e0 con 0x7f1a400cfae0 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a2c003520 con 0x7f1a400cfae0 2024-03-04T12:12:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 264984494 0 0) 0x7f1a2c003520 con 0x7f1a400cfae0 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 >> v2:172.21.15.191:3300/0 conn(0x7f1a40154a80 msgr2=0x7f1a4015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 --2- 172.21.15.148:0/66779662 >> v2:172.21.15.191:3300/0 conn(0x7f1a40154a80 0x7f1a4015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 >> v1:172.21.15.191:6789/0 conn(0x7f1a400cf650 legacy=0x7f1a401616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400cff70 con 0x7f1a400cfae0 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f1a380019f0 con 0x7f1a400cfae0 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 >> v1:172.21.15.148:6789/0 conn(0x7f1a400cfae0 legacy=0x7f1a40161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0039b0 0x7f1a2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 0x7f1a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0051c0 0x7f1a2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40071df0 con 0x7f1a2c004800 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c00c060 con 0x7f1a2c0039b0 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c00c260 con 0x7f1a2c0051c0 2024-03-04T12:12:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 0x7f1a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a454f5640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0039b0 0x7f1a2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a45cf6640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0051c0 0x7f1a2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 -- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0051c0 msgr2=0x7f1a2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0051c0 0x7f1a2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 -- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0039b0 msgr2=0x7f1a2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0039b0 0x7f1a2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a44cf4640 1 -- 172.21.15.148:0/66779662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c003520 con 0x7f1a2c004800 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a45cf6640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0051c0 0x7f1a2c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.759+0000 7f1a45cf6640 1 -- 172.21.15.148:0/66779662 reap_dead start 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.760+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 0x7f1a2c004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a30001750 tx=0x7f1a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a8ff717f94a5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.760+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30001370 con 0x7f1a2c004800 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.760+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a300052d0 con 0x7f1a2c004800 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.760+0000 7f1a37fff640 1 -- 172.21.15.148:0/66779662 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30003830 con 0x7f1a2c004800 2024-03-04T12:12:45.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 -- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 msgr2=0x7f1a2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 0x7f1a2c004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a30001750 tx=0x7f1a2c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 -- 172.21.15.148:0/66779662 shutdown_connections 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 --2- 172.21.15.148:0/66779662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c004800 0x7f1a2c004c80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 -- 172.21.15.148:0/66779662 >> 172.21.15.148:0/66779662 conn(0x7f1a4006f5f0 msgr2=0x7f1a4006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 -- 172.21.15.148:0/66779662 shutdown_connections 2024-03-04T12:12:45.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.761+0000 7f1a47759640 1 -- 172.21.15.148:0/66779662 wait complete. 2024-03-04T12:12:45.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.762+0000 7f1a47759640 1 Processor -- start 2024-03-04T12:12:45.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.762+0000 7f1a47759640 1 -- start start 2024-03-04T12:12:45.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c004800 0x7f1a40161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a4015ed20 0x7f1a400c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400ce060 con 0x7f1a4015e8a0 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400cdd80 con 0x7f1a2c004800 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400ce340 con 0x7f1a4015ed20 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58906/0 (socket says 172.21.15.148:58906) 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 -- 172.21.15.148:0/2670522210 learned_addr learned my addr 172.21.15.148:0/2670522210 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a45cf6640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a4015ed20 0x7f1a400c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a454f5640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c004800 0x7f1a40161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 -- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a4015ed20 msgr2=0x7f1a400c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a4015ed20 0x7f1a400c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 -- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c004800 msgr2=0x7f1a40161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c004800 0x7f1a40161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 -- 172.21.15.148:0/2670522210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40161940 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a44cf4640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a30000c00 tx=0x7f1a30001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30041020 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a30005480 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40146610 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.763+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30007a30 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.764+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a40160920 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.765+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a40074f60 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.765+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1a30044020 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.765+0000 7f1a35ffb640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 0x7f1a2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.766+0000 7f1a454f5640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 0x7f1a2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.766+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1a300abf00 con 0x7f1a4015e8a0 2024-03-04T12:12:45.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.766+0000 7f1a454f5640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 0x7f1a2406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c40 tx=0x7f1a3c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:45.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.770+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a3004b030 con 0x7f1a4015e8a0 2024-03-04T12:12:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.943+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a40160b60 con 0x7f1a4015e8a0 2024-03-04T12:12:45.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.944+0000 7f1a35ffb640 1 -- 172.21.15.148:0/2670522210 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f1a3007a8f0 con 0x7f1a4015e8a0 2024-03-04T12:12:45.946 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:45.946 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 msgr2=0x7f1a2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 0x7f1a2406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c40 tx=0x7f1a3c009190 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 msgr2=0x7f1a400c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a30000c00 tx=0x7f1a30001120 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 shutdown_connections 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1a2406b500 0x7f1a2406d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a4015ed20 0x7f1a400c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4015e8a0 0x7f1a400c3fb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 --2- 172.21.15.148:0/2670522210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c004800 0x7f1a40161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 >> 172.21.15.148:0/2670522210 conn(0x7f1a4006f5f0 msgr2=0x7f1a400b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 shutdown_connections 2024-03-04T12:12:45.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:45.946+0000 7f1a47759640 1 -- 172.21.15.148:0/2670522210 wait complete. 2024-03-04T12:12:45.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:45.959 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.061+0000 7fdd2b91b640 1 Processor -- start 2024-03-04T12:12:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 -- start start 2024-03-04T12:12:46.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd2406b700 0x7fdd2406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd240cfa40 0x7fdd240cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2406bb60 con 0x7fdd2415df00 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd240701d0 con 0x7fdd240cfa40 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd2b91b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd240704b0 con 0x7fdd2406b700 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd29eb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd240cfa40 0x7fdd240cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd296b7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdd2415df00 0x7fdd240cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50204/0 (socket says 172.21.15.148:50204) 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.062+0000 7fdd296b7640 1 -- 172.21.15.148:0/2979574698 learned_addr learned my addr 172.21.15.148:0/2979574698 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd29eb8640 1 -- 172.21.15.148:0/2979574698 >> v2:172.21.15.191:3300/0 conn(0x7fdd2406b700 msgr2=0x7fdd2406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd29eb8640 1 --2- 172.21.15.148:0/2979574698 >> v2:172.21.15.191:3300/0 conn(0x7fdd2406b700 0x7fdd2406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd29eb8640 1 -- 172.21.15.148:0/2979574698 >> v1:172.21.15.148:6789/0 conn(0x7fdd2415df00 legacy=0x7fdd240cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd1bfff640 1 -- 172.21.15.148:0/2979574698 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372190305 0 0) 0x7fdd2406bb60 con 0x7fdd2415df00 2024-03-04T12:12:46.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd29eb8640 1 -- 172.21.15.148:0/2979574698 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240cc8e0 con 0x7fdd240cfa40 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.063+0000 7fdd29eb8640 1 --2- 172.21.15.148:0/2979574698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd240cfa40 0x7fdd240cfed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd240719e0 tx=0x7fdd1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e567520eaedee123 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd1bfff640 1 -- 172.21.15.148:0/2979574698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c035020 con 0x7fdd240cfa40 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd1bfff640 1 -- 172.21.15.148:0/2979574698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c031830 con 0x7fdd240cfa40 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd1bfff640 1 -- 172.21.15.148:0/2979574698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031c00 con 0x7fdd240cfa40 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 -- 172.21.15.148:0/2979574698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd240cfa40 msgr2=0x7fdd240cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/2979574698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd240cfa40 0x7fdd240cfed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd240719e0 tx=0x7fdd1c02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 -- 172.21.15.148:0/2979574698 shutdown_connections 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/2979574698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd240cfa40 0x7fdd240cfed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/2979574698 >> v2:172.21.15.191:3300/0 conn(0x7fdd2406b700 0x7fdd2406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 -- 172.21.15.148:0/2979574698 >> 172.21.15.148:0/2979574698 conn(0x7fdd240b2250 msgr2=0x7fdd240b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 -- 172.21.15.148:0/2979574698 shutdown_connections 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.064+0000 7fdd2b91b640 1 -- 172.21.15.148:0/2979574698 wait complete. 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.065+0000 7fdd2b91b640 1 Processor -- start 2024-03-04T12:12:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.065+0000 7fdd2b91b640 1 -- start start 2024-03-04T12:12:46.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd240cfa40 0x7fdd24153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd2415df00 0x7fdd24153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240d0410 con 0x7fdd2406b700 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240ccea0 con 0x7fdd240cfa40 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd2b91b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240cd3c0 con 0x7fdd2415df00 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd296b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd296b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58928/0 (socket says 172.21.15.148:58928) 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.066+0000 7fdd296b7640 1 -- 172.21.15.148:0/3116264917 learned_addr learned my addr 172.21.15.148:0/3116264917 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 -- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd2415df00 msgr2=0x7fdd24153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd2415df00 0x7fdd24153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 -- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd240cfa40 msgr2=0x7fdd24153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd240cfa40 0x7fdd24153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 -- 172.21.15.148:0/3116264917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2409c9b0 con 0x7fdd2406b700 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd296b7640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7c0 tx=0x7fdd2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20018070 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000a640 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd240c62f0 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000aa30 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.067+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd240c68b0 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.068+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd24074f60 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.068+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd2001b070 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.069+0000 7fdd19ffb640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 0x7fdd0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.069+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdd20089ba0 con 0x7fdd2406b700 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.069+0000 7fdd28eb6640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 0x7fdd0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.069+0000 7fdd28eb6640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 0x7fdd0806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd2415d320 tx=0x7fdd14006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.073+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd200586b0 con 0x7fdd2406b700 2024-03-04T12:12:46.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.243+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd24066e00 con 0x7fdd2406b700 2024-03-04T12:12:46.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.244+0000 7fdd19ffb640 1 -- 172.21.15.148:0/3116264917 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd20057fb0 con 0x7fdd2406b700 2024-03-04T12:12:46.245 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:46.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.245+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 msgr2=0x7fdd0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.245+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 0x7fdd0806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd2415d320 tx=0x7fdd14006c90 comp rx=0 tx=0).stop 2024-03-04T12:12:46.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 msgr2=0x7fdd2409a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7c0 tx=0x7fdd2000cc80 comp rx=0 tx=0).stop 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 shutdown_connections 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd0806b4d0 0x7fdd0806d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd2415df00 0x7fdd24153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd240cfa40 0x7fdd24153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 --2- 172.21.15.148:0/3116264917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406b700 0x7fdd2409a8b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 >> 172.21.15.148:0/3116264917 conn(0x7fdd240b2250 msgr2=0x7fdd240d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 shutdown_connections 2024-03-04T12:12:46.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.246+0000 7fdd2b91b640 1 -- 172.21.15.148:0/3116264917 wait complete. 2024-03-04T12:12:46.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:46.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.360+0000 7fee23fcd640 1 Processor -- start 2024-03-04T12:12:46.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.361+0000 7fee23fcd640 1 -- start start 2024-03-04T12:12:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.361+0000 7fee23fcd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 0x7fee1c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.361+0000 7fee23fcd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee1c0cf660 0x7fee1c0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.361+0000 7fee23fcd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 0x7fee1c161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee23fcd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0d0000 con 0x7fee1c0cf660 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee23fcd640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0cc920 con 0x7fee1c1616e0 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee23fcd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c162450 con 0x7fee1c15da20 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee1c0cf660 0x7fee1c0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee1c0cf660 0x7fee1c0cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58944/0 (socket says 172.21.15.148:58944) 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 -- 172.21.15.148:0/1804166004 learned_addr learned my addr 172.21.15.148:0/1804166004 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 -- 172.21.15.148:0/1804166004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 msgr2=0x7fee1c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21d69640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 0x7fee1c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee2256a640 1 --2- 172.21.15.148:0/1804166004 >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 0x7fee1c161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 0x7fee1c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 -- 172.21.15.148:0/1804166004 >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 msgr2=0x7fee1c161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 --2- 172.21.15.148:0/1804166004 >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 0x7fee1c161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21568640 1 -- 172.21.15.148:0/1804166004 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0cc2d0 con 0x7fee1c0cf660 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee21d69640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 0x7fee1c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.362+0000 7fee2256a640 1 --2- 172.21.15.148:0/1804166004 >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 0x7fee1c161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee21568640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c0cf660 0x7fee1c0cfac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0d1340 tx=0x7fee1802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883c108d9dd788b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee20d67640 1 -- 172.21.15.148:0/1804166004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18035020 con 0x7fee1c0cf660 2024-03-04T12:12:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee20d67640 1 -- 172.21.15.148:0/1804166004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1802d040 con 0x7fee1c0cf660 2024-03-04T12:12:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee20d67640 1 -- 172.21.15.148:0/1804166004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18031a80 con 0x7fee1c0cf660 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 -- 172.21.15.148:0/1804166004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c0cf660 msgr2=0x7fee1c0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c0cf660 0x7fee1c0cfac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0d1340 tx=0x7fee1802b900 comp rx=0 tx=0).stop 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 -- 172.21.15.148:0/1804166004 shutdown_connections 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 --2- 172.21.15.148:0/1804166004 >> v2:172.21.15.191:3300/0 conn(0x7fee1c1616e0 0x7fee1c161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c0cf660 0x7fee1c0cfac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 --2- 172.21.15.148:0/1804166004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c15da20 0x7fee1c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.363+0000 7fee23fcd640 1 -- 172.21.15.148:0/1804166004 >> 172.21.15.148:0/1804166004 conn(0x7fee1c0b33a0 msgr2=0x7fee1c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.364+0000 7fee23fcd640 1 -- 172.21.15.148:0/1804166004 shutdown_connections 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.364+0000 7fee23fcd640 1 -- 172.21.15.148:0/1804166004 wait complete. 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 Processor -- start 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 -- start start 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee1c0cf660 0x7fee1c160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c1616e0 0x7fee1c1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0cbf80 con 0x7fee1c15da20 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0cd2f0 con 0x7fee1c0cf660 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee23fcd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c15de00 con 0x7fee1c1616e0 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee21568640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee21568640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58950/0 (socket says 172.21.15.148:58950) 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.365+0000 7fee21568640 1 -- 172.21.15.148:0/3031529174 learned_addr learned my addr 172.21.15.148:0/3031529174 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 -- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c1616e0 msgr2=0x7fee1c1453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee2256a640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c1616e0 0x7fee1c1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c1616e0 0x7fee1c1453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21d69640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee1c0cf660 0x7fee1c160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 -- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee1c0cf660 msgr2=0x7fee1c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee1c0cf660 0x7fee1c160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 -- 172.21.15.148:0/3031529174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c15fce0 con 0x7fee1c15da20 2024-03-04T12:12:46.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee21568640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee1802b940 tx=0x7fee180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18040070 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee18039c30 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18035040 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c145930 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.366+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c15ff80 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.367+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c074f60 con 0x7fee1c15da20 2024-03-04T12:12:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.367+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fee1804b060 con 0x7fee1c15da20 2024-03-04T12:12:46.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.368+0000 7fee0e7fc640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 0x7fee0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.368+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fee180ac3e0 con 0x7fee1c15da20 2024-03-04T12:12:46.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.368+0000 7fee21d69640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 0x7fee0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.368+0000 7fee21d69640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 0x7fee0006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee10008850 tx=0x7fee10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.372+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee1807ae60 con 0x7fee1c15da20 2024-03-04T12:12:46.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.544+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee1c06e620 con 0x7fee1c15da20 2024-03-04T12:12:46.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.544+0000 7fee0e7fc640 1 -- 172.21.15.148:0/3031529174 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee1807a760 con 0x7fee1c15da20 2024-03-04T12:12:46.546 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 msgr2=0x7fee0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 0x7fee0006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee10008850 tx=0x7fee10008040 comp rx=0 tx=0).stop 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 msgr2=0x7fee1c0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee1802b940 tx=0x7fee180046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 shutdown_connections 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fee0006b4d0 0x7fee0006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee1c1616e0 0x7fee1c1453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee1c15da20 0x7fee1c0c7d10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 --2- 172.21.15.148:0/3031529174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee1c0cf660 0x7fee1c160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 >> 172.21.15.148:0/3031529174 conn(0x7fee1c0b33a0 msgr2=0x7fee1c06bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 shutdown_connections 2024-03-04T12:12:46.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.546+0000 7fee23fcd640 1 -- 172.21.15.148:0/3031529174 wait complete. 2024-03-04T12:12:46.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:46.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 Processor -- start 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 -- start start 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe8380d34a0 0x7fe838161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8380d2400 con 0x7fe8380b9410 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8380d26e0 con 0x7fe8380b5700 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.661+0000 7fe83cdda640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380d29c0 con 0x7fe8380d34a0 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe836d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe8380b9410 0x7fe838162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50212/0 (socket says 172.21.15.148:50212) 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe836d9d640 1 -- 172.21.15.148:0/144541540 learned_addr learned my addr 172.21.15.148:0/144541540 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe82ffff640 1 --2- 172.21.15.148:0/144541540 >> v2:172.21.15.191:3300/0 conn(0x7fe8380d34a0 0x7fe838161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe835d9b640 1 -- 172.21.15.148:0/144541540 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421088352 0 0) 0x7fe8380d2400 con 0x7fe8380b9410 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe835d9b640 1 -- 172.21.15.148:0/144541540 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe824003520 con 0x7fe8380b9410 2024-03-04T12:12:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe82ffff640 1 -- 172.21.15.148:0/144541540 >> v1:172.21.15.191:6789/0 conn(0x7fe8380b5700 legacy=0x7fe8380d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe82ffff640 1 -- 172.21.15.148:0/144541540 >> v1:172.21.15.148:6789/0 conn(0x7fe8380b9410 legacy=0x7fe838162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.662+0000 7fe82ffff640 1 -- 172.21.15.148:0/144541540 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8380d2ca0 con 0x7fe8380d34a0 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.663+0000 7fe82ffff640 1 --2- 172.21.15.148:0/144541540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe838161be0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097a0 tx=0x7fe82002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d400942994838c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.663+0000 7fe835d9b640 1 -- 172.21.15.148:0/144541540 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820035020 con 0x7fe8380d34a0 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.663+0000 7fe835d9b640 1 -- 172.21.15.148:0/144541540 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe820031830 con 0x7fe8380d34a0 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.663+0000 7fe835d9b640 1 -- 172.21.15.148:0/144541540 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820031be0 con 0x7fe8380d34a0 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 -- 172.21.15.148:0/144541540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 msgr2=0x7fe838161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 --2- 172.21.15.148:0/144541540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe838161be0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097a0 tx=0x7fe82002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 -- 172.21.15.148:0/144541540 shutdown_connections 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 --2- 172.21.15.148:0/144541540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe838161be0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 -- 172.21.15.148:0/144541540 >> 172.21.15.148:0/144541540 conn(0x7fe8380d0490 msgr2=0x7fe8380d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 -- 172.21.15.148:0/144541540 shutdown_connections 2024-03-04T12:12:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.664+0000 7fe83cdda640 1 -- 172.21.15.148:0/144541540 wait complete. 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.665+0000 7fe83cdda640 1 Processor -- start 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.665+0000 7fe83cdda640 1 -- start start 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8380b9410 0x7fe83815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe83815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b5ae0 con 0x7fe8380b5700 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838162a40 con 0x7fe8380d34a0 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b5c60 con 0x7fe8380b9410 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58958/0 (socket says 172.21.15.148:58958) 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 -- 172.21.15.148:0/743486888 learned_addr learned my addr 172.21.15.148:0/743486888 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe82ffff640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8380b9410 0x7fe83815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe836d9d640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe83815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 -- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8380b9410 msgr2=0x7fe83815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8380b9410 0x7fe83815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 -- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 msgr2=0x7fe83815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe83815bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 -- 172.21.15.148:0/743486888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8381f3740 con 0x7fe8380b5700 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe836d9d640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe83815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83659c640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c400 tx=0x7fe82800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828017070 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8280052d0 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8380cbcd0 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828003830 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.666+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8380cbf70 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.668+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe838074f60 con 0x7fe8380b5700 2024-03-04T12:12:46.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.668+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe82801a070 con 0x7fe8380b5700 2024-03-04T12:12:46.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.668+0000 7fe82effd640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 0x7fe81806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.668+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe828089a20 con 0x7fe8380b5700 2024-03-04T12:12:46.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.668+0000 7fe82ffff640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 0x7fe81806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.669+0000 7fe82ffff640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 0x7fe81806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe820008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.673+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe828058420 con 0x7fe8380b5700 2024-03-04T12:12:46.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.842+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8380b7110 con 0x7fe8380b5700 2024-03-04T12:12:46.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.842+0000 7fe82effd640 1 -- 172.21.15.148:0/743486888 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe828057d20 con 0x7fe8380b5700 2024-03-04T12:12:46.844 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:46.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 msgr2=0x7fe81806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 0x7fe81806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe820008030 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 msgr2=0x7fe8380b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c400 tx=0x7fe82800c8c0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 shutdown_connections 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe81806b4d0 0x7fe81806d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8380d34a0 0x7fe83815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8380b9410 0x7fe83815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 --2- 172.21.15.148:0/743486888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8380b5700 0x7fe8380b7fa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.844+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 >> 172.21.15.148:0/743486888 conn(0x7fe8380d0490 msgr2=0x7fe8380d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.845+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 shutdown_connections 2024-03-04T12:12:46.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.845+0000 7fe83cdda640 1 -- 172.21.15.148:0/743486888 wait complete. 2024-03-04T12:12:46.857 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:46.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.962+0000 7fc2fdd7d640 1 Processor -- start 2024-03-04T12:12:46.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 -- start start 2024-03-04T12:12:46.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f80cb290 con 0x7fc2f80cfa40 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f80701d0 con 0x7fc2f806fa80 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.963+0000 7fc2fdd7d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80704b0 con 0x7fc2f815df00 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2eeffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:33586/0 (socket says 172.21.15.148:33586) 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2eeffd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:36510/0 (socket says 172.21.15.148:36510) 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 -- 172.21.15.148:0/3897159037 learned_addr learned my addr 172.21.15.148:0/3897159037 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f7fff640 1 --1- 172.21.15.148:0/3897159037 >> v1:172.21.15.191:6789/0 conn(0x7fc2f80cfa40 0x7fc2f80cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:54820/0 (socket says 172.21.15.148:54820) 2024-03-04T12:12:46.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 -- 172.21.15.148:0/3897159037 >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 msgr2=0x7fc2f80cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/3897159037 >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 -- 172.21.15.148:0/3897159037 >> v1:172.21.15.191:6789/0 conn(0x7fc2f80cfa40 legacy=0x7fc2f80cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2f77fe640 1 -- 172.21.15.148:0/3897159037 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80cc8e0 con 0x7fc2f815df00 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.964+0000 7fc2eeffd640 1 --2- 172.21.15.148:0/3897159037 >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.965+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/3897159037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3303759c042f8ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.965+0000 7fc2f6ffd640 1 -- 172.21.15.148:0/3897159037 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8035020 con 0x7fc2f815df00 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.965+0000 7fc2f6ffd640 1 -- 172.21.15.148:0/3897159037 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e8031640 con 0x7fc2f815df00 2024-03-04T12:12:46.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.965+0000 7fc2f6ffd640 1 -- 172.21.15.148:0/3897159037 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e802d040 con 0x7fc2f815df00 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/3897159037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 msgr2=0x7fc2f806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/3897159037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e802b8f0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/3897159037 shutdown_connections 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/3897159037 >> v2:172.21.15.191:3300/0 conn(0x7fc2f806fa80 0x7fc2f80cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/3897159037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f815df00 0x7fc2f806b700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/3897159037 >> 172.21.15.148:0/3897159037 conn(0x7fc2f80b2250 msgr2=0x7fc2f80b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/3897159037 shutdown_connections 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.966+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/3897159037 wait complete. 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.967+0000 7fc2fdd7d640 1 Processor -- start 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.967+0000 7fc2fdd7d640 1 -- start start 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 0x7fc2f815e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2f80cfa40 0x7fc2f815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f806fee0 con 0x7fc2f815df00 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80ccea0 con 0x7fc2f80cfa40 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2fdd7d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80cd3c0 con 0x7fc2f806fa80 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58970/0 (socket says 172.21.15.148:58970) 2024-03-04T12:12:46.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 -- 172.21.15.148:0/1091851903 learned_addr learned my addr 172.21.15.148:0/1091851903 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 0x7fc2f815e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2eeffd640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2f80cfa40 0x7fc2f815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 -- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 msgr2=0x7fc2f815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 0x7fc2f815e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 -- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2f80cfa40 msgr2=0x7fc2f815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2f80cfa40 0x7fc2f815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 -- 172.21.15.148:0/1091851903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80b46e0 con 0x7fc2f815df00 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 0x7fc2f815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f7fff640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000c7b0 tx=0x7fc2f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.968+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0017070 con 0x7fc2f815df00 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.969+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000a640 con 0x7fc2f815df00 2024-03-04T12:12:46.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.969+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80b4950 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.969+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000a9f0 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.969+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f815d0f0 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.969+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f8074f60 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.970+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc2f001a070 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.970+0000 7fc2f4ff9640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.970+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc2f0089a50 con 0x7fc2f815df00 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.970+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:46.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.971+0000 7fc2f77fe640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8036000 tx=0x7fc2e803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:46.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:46.975+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2f0057f10 con 0x7fc2f815df00 2024-03-04T12:12:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.148+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2f806d460 con 0x7fc2f815df00 2024-03-04T12:12:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.148+0000 7fc2f4ff9640 1 -- 172.21.15.148:0/1091851903 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc2f005bfb0 con 0x7fc2f815df00 2024-03-04T12:12:47.150 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 msgr2=0x7fc2d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8036000 tx=0x7fc2e803c040 comp rx=0 tx=0).stop 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 msgr2=0x7fc2f806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000c7b0 tx=0x7fc2f000cc70 comp rx=0 tx=0).stop 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 shutdown_connections 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f815df00 0x7fc2f806e520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2f80cfa40 0x7fc2f815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 --2- 172.21.15.148:0/1091851903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2f806fa80 0x7fc2f815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 >> 172.21.15.148:0/1091851903 conn(0x7fc2f80b2250 msgr2=0x7fc2f80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 shutdown_connections 2024-03-04T12:12:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.150+0000 7fc2fdd7d640 1 -- 172.21.15.148:0/1091851903 wait complete. 2024-03-04T12:12:47.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:47.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.263+0000 7f38ced59640 1 Processor -- start 2024-03-04T12:12:47.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.263+0000 7f38ced59640 1 -- start start 2024-03-04T12:12:47.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ced59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38c815b030 0x7f38c815c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ced59640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f38c806d3b0 0x7f38c815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ced59640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c815e2d0 con 0x7f38c815b410 2024-03-04T12:12:47.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ced59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80b2720 con 0x7f38c815b030 2024-03-04T12:12:47.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ced59640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c806d8b0 con 0x7f38c806d3b0 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38c815b030 0x7f38c815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38c815b030 0x7f38c815c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58972/0 (socket says 172.21.15.148:58972) 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 -- 172.21.15.148:0/2607776533 learned_addr learned my addr 172.21.15.148:0/2607776533 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38c7fff640 1 --2- 172.21.15.148:0/2607776533 >> v2:172.21.15.191:3300/0 conn(0x7f38c806d3b0 0x7f38c815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 -- 172.21.15.148:0/2607776533 >> v2:172.21.15.191:3300/0 conn(0x7f38c806d3b0 msgr2=0x7f38c815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 --2- 172.21.15.148:0/2607776533 >> v2:172.21.15.191:3300/0 conn(0x7f38c806d3b0 0x7f38c815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 -- 172.21.15.148:0/2607776533 >> v1:172.21.15.191:6789/0 conn(0x7f38c815b410 legacy=0x7f38c815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.264+0000 7f38ccaf5640 1 -- 172.21.15.148:0/2607776533 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c806db90 con 0x7f38c815b030 2024-03-04T12:12:47.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.265+0000 7f38ccaf5640 1 --2- 172.21.15.148:0/2607776533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b030 0x7f38c815c120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f38c80cd300 tx=0x7f38bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e1d7adbcbd05c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.265+0000 7f38c77fe640 1 -- 172.21.15.148:0/2607776533 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc02e070 con 0x7f38c815b030 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.265+0000 7f38c77fe640 1 -- 172.21.15.148:0/2607776533 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc037070 con 0x7f38c815b030 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.265+0000 7f38c77fe640 1 -- 172.21.15.148:0/2607776533 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc031a00 con 0x7f38c815b030 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 -- 172.21.15.148:0/2607776533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b030 msgr2=0x7f38c815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 --2- 172.21.15.148:0/2607776533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b030 0x7f38c815c120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f38c80cd300 tx=0x7f38bc02b880 comp rx=0 tx=0).stop 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 -- 172.21.15.148:0/2607776533 shutdown_connections 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 --2- 172.21.15.148:0/2607776533 >> v2:172.21.15.191:3300/0 conn(0x7f38c806d3b0 0x7f38c815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 --2- 172.21.15.148:0/2607776533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b030 0x7f38c815c120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 -- 172.21.15.148:0/2607776533 >> 172.21.15.148:0/2607776533 conn(0x7f38c806acd0 msgr2=0x7f38c806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 -- 172.21.15.148:0/2607776533 shutdown_connections 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.266+0000 7f38ced59640 1 -- 172.21.15.148:0/2607776533 wait complete. 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.267+0000 7f38ced59640 1 Processor -- start 2024-03-04T12:12:47.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.267+0000 7f38ced59640 1 -- start start 2024-03-04T12:12:47.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c806d3b0 0x7f38c815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c815b030 0x7f38c81390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c806e690 con 0x7f38c815b410 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c806e150 con 0x7f38c815b030 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ced59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c806e810 con 0x7f38c806d3b0 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58980/0 (socket says 172.21.15.148:58980) 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 -- 172.21.15.148:0/4024509940 learned_addr learned my addr 172.21.15.148:0/4024509940 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38c7fff640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c815b030 0x7f38c81390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38ccaf5640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c806d3b0 0x7f38c815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 -- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c806d3b0 msgr2=0x7f38c815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c806d3b0 0x7f38c815c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 -- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c815b030 msgr2=0x7f38c81390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c815b030 0x7f38c81390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 -- 172.21.15.148:0/4024509940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8154b60 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.268+0000 7f38cd2f6640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c7a0 tx=0x7f38c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.269+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0003b70 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.269+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0003d30 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.269+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c81542c0 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.269+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c000aa90 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.269+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c8154880 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.270+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c8074f60 con 0x7f38c815b410 2024-03-04T12:12:47.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.270+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f38c000ac30 con 0x7f38c815b410 2024-03-04T12:12:47.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.271+0000 7f38c57fa640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 0x7f38ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.271+0000 7f38ccaf5640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 0x7f38ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.271+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f38c008a400 con 0x7f38c815b410 2024-03-04T12:12:47.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.271+0000 7f38ccaf5640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 0x7f38ac06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38bc036000 tx=0x7f38bc03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.275+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38c0058830 con 0x7f38c815b410 2024-03-04T12:12:47.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.458+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38c8154dd0 con 0x7f38c815b410 2024-03-04T12:12:47.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.458+0000 7f38c57fa640 1 -- 172.21.15.148:0/4024509940 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f38c005c8d0 con 0x7f38c815b410 2024-03-04T12:12:47.460 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:47.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 msgr2=0x7f38ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 0x7f38ac06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38bc036000 tx=0x7f38bc03d040 comp rx=0 tx=0).stop 2024-03-04T12:12:47.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 msgr2=0x7f38c8139630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c7a0 tx=0x7f38c000cc60 comp rx=0 tx=0).stop 2024-03-04T12:12:47.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 shutdown_connections 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f38ac06b4d0 0x7f38ac06d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c815b410 0x7f38c8139630 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c815b030 0x7f38c81390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 --2- 172.21.15.148:0/4024509940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c806d3b0 0x7f38c815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 >> 172.21.15.148:0/4024509940 conn(0x7f38c806acd0 msgr2=0x7f38c80b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 shutdown_connections 2024-03-04T12:12:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.460+0000 7f38ced59640 1 -- 172.21.15.148:0/4024509940 wait complete. 2024-03-04T12:12:47.473 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-04T12:12:47.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:47.473 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-04T12:12:47.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:12:47.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:47.507 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-04T12:12:47.568 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:47.568 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:47.583 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:47.583 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:47.638 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:12:47.638 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:47.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:47.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.832+0000 7f6817c4b640 1 Processor -- start 2024-03-04T12:12:47.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.833+0000 7f6817c4b640 1 -- start start 2024-03-04T12:12:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.833+0000 7f6817c4b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f68100b2e60 0x7f68100b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.833+0000 7f6817c4b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68100b5470 0x7f68100b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.833+0000 7f6817c4b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681015e1e0 con 0x7f681015de00 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.833+0000 7f6817c4b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b98c0 con 0x7f68100b5470 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f6817c4b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b32c0 con 0x7f68100b2e60 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68100b5470 0x7f68100b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68100b5470 0x7f68100b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58996/0 (socket says 172.21.15.148:58996) 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 -- 172.21.15.148:0/1406746293 learned_addr learned my addr 172.21.15.148:0/1406746293 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68151e6640 1 --2- 172.21.15.148:0/1406746293 >> v2:172.21.15.191:3300/0 conn(0x7f68100b2e60 0x7f68100b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 -- 172.21.15.148:0/1406746293 >> v2:172.21.15.191:3300/0 conn(0x7f68100b2e60 msgr2=0x7f68100b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 --2- 172.21.15.148:0/1406746293 >> v2:172.21.15.191:3300/0 conn(0x7f68100b2e60 0x7f68100b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 -- 172.21.15.148:0/1406746293 >> v1:172.21.15.191:6789/0 conn(0x7f681015de00 legacy=0x7f68100b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.834+0000 7f68161e8640 1 -- 172.21.15.148:0/1406746293 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100d1c10 con 0x7f68100b5470 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f68161e8640 1 --2- 172.21.15.148:0/1406746293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68100b5470 0x7f68100b5900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6810161830 tx=0x7f680802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ee1b1944b2ed87 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f68149e5640 1 -- 172.21.15.148:0/1406746293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808034070 con 0x7f68100b5470 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f68149e5640 1 -- 172.21.15.148:0/1406746293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808030860 con 0x7f68100b5470 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f68149e5640 1 -- 172.21.15.148:0/1406746293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808030c30 con 0x7f68100b5470 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 -- 172.21.15.148:0/1406746293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68100b5470 msgr2=0x7f68100b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 --2- 172.21.15.148:0/1406746293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68100b5470 0x7f68100b5900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6810161830 tx=0x7f680802b9a0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 -- 172.21.15.148:0/1406746293 shutdown_connections 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 --2- 172.21.15.148:0/1406746293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68100b5470 0x7f68100b5900 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 --2- 172.21.15.148:0/1406746293 >> v2:172.21.15.191:3300/0 conn(0x7f68100b2e60 0x7f68100b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.835+0000 7f6817c4b640 1 -- 172.21.15.148:0/1406746293 >> 172.21.15.148:0/1406746293 conn(0x7f68100d0470 msgr2=0x7f68100d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:47.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.836+0000 7f6817c4b640 1 -- 172.21.15.148:0/1406746293 shutdown_connections 2024-03-04T12:12:47.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.836+0000 7f6817c4b640 1 -- 172.21.15.148:0/1406746293 wait complete. 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 Processor -- start 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 -- start start 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68100b2e60 0x7f68100c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68100b5470 0x7f68100b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100d21d0 con 0x7f681015de00 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100d2820 con 0x7f68100b5470 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f6817c4b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100d24b0 con 0x7f68100b2e60 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.837+0000 7f68161e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59004/0 (socket says 172.21.15.148:59004) 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 -- 172.21.15.148:0/2286676108 learned_addr learned my addr 172.21.15.148:0/2286676108 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68151e6640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68100b5470 0x7f68100b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68159e7640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68100b2e60 0x7f68100c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 -- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68100b2e60 msgr2=0x7f68100c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68100b2e60 0x7f68100c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 -- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68100b5470 msgr2=0x7f68100b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68100b5470 0x7f68100b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 -- 172.21.15.148:0/2286676108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100c9db0 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68161e8640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f680802b9e0 tx=0x7f6808004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808048070 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808030660 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100ca050 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808034040 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.838+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681015f460 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.839+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68100ca220 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.839+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f680804b070 con 0x7f681015de00 2024-03-04T12:12:47.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.840+0000 7f68067fc640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.840+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f68080ac820 con 0x7f681015de00 2024-03-04T12:12:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.840+0000 7f68159e7640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.840+0000 7f68159e7640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 0x7f67f406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f680c001ca0 tx=0x7f680c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:47.844+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f680807b3c0 con 0x7f681015de00 2024-03-04T12:12:48.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.019+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68100c9b10 con 0x7f681015de00 2024-03-04T12:12:48.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.020+0000 7f68067fc640 1 -- 172.21.15.148:0/2286676108 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f680807acc0 con 0x7f681015de00 2024-03-04T12:12:48.022 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:48.022 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 msgr2=0x7f67f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 0x7f67f406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f680c001ca0 tx=0x7f680c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 msgr2=0x7f68100b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f680802b9e0 tx=0x7f6808004730 comp rx=0 tx=0).stop 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 shutdown_connections 2024-03-04T12:12:48.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681015de00 0x7f68100b7750 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68100b5470 0x7f68100b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 --2- 172.21.15.148:0/2286676108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68100b2e60 0x7f68100c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 >> 172.21.15.148:0/2286676108 conn(0x7f68100d0470 msgr2=0x7f681006c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 shutdown_connections 2024-03-04T12:12:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.022+0000 7f6817c4b640 1 -- 172.21.15.148:0/2286676108 wait complete. 2024-03-04T12:12:48.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:48.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:48.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.138+0000 7f4a7f00c640 1 Processor -- start 2024-03-04T12:12:48.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.138+0000 7f4a7f00c640 1 -- start start 2024-03-04T12:12:48.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7f00c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 0x7f4a780ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7f00c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a78161df0 0x7f4a780cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7f00c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a78162450 con 0x7f4a780cf660 2024-03-04T12:12:48.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7f00c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a780d0000 con 0x7f4a78161df0 2024-03-04T12:12:48.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7f00c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780cc920 con 0x7f4a7815da20 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7cda8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 0x7f4a780ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a7d5a9640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a78161df0 0x7f4a780cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a77fff640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4a780cf660 0x7f4a781616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:54856/0 (socket says 172.21.15.148:54856) 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a77fff640 1 -- 172.21.15.148:0/3398645134 learned_addr learned my addr 172.21.15.148:0/3398645134 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a777fe640 1 -- 172.21.15.148:0/3398645134 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196278210 0 0) 0x7f4a78162450 con 0x7f4a780cf660 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.139+0000 7f4a777fe640 1 -- 172.21.15.148:0/3398645134 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a68003520 con 0x7f4a780cf660 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7d5a9640 1 -- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 msgr2=0x7f4a780ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7d5a9640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 0x7f4a780ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7d5a9640 1 -- 172.21.15.148:0/3398645134 >> v1:172.21.15.191:6789/0 conn(0x7f4a780cf660 legacy=0x7f4a781616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7d5a9640 1 -- 172.21.15.148:0/3398645134 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780cc2d0 con 0x7f4a78161df0 2024-03-04T12:12:48.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7cda8640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 0x7f4a780ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:48.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a7d5a9640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a78161df0 0x7f4a780cfac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009d90 tx=0x7f4a7002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e9927e697c6da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a777fe640 1 -- 172.21.15.148:0/3398645134 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70039070 con 0x7f4a78161df0 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a777fe640 1 -- 172.21.15.148:0/3398645134 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7002cc50 con 0x7f4a78161df0 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.140+0000 7f4a777fe640 1 -- 172.21.15.148:0/3398645134 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70034440 con 0x7f4a78161df0 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 -- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a78161df0 msgr2=0x7f4a780cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a78161df0 0x7f4a780cfac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009d90 tx=0x7f4a7002bea0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 -- 172.21.15.148:0/3398645134 shutdown_connections 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a78161df0 0x7f4a780cfac0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/3398645134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a7815da20 0x7f4a780ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 -- 172.21.15.148:0/3398645134 >> 172.21.15.148:0/3398645134 conn(0x7f4a780b33a0 msgr2=0x7f4a780b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 -- 172.21.15.148:0/3398645134 shutdown_connections 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.141+0000 7f4a7f00c640 1 -- 172.21.15.148:0/3398645134 wait complete. 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.142+0000 7f4a7f00c640 1 Processor -- start 2024-03-04T12:12:48.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.142+0000 7f4a7f00c640 1 -- start start 2024-03-04T12:12:48.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 0x7f4a78157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 0x7f4a781f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780cbf80 con 0x7f4a780cf660 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780cd2f0 con 0x7f4a7815da20 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7f00c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7815de00 con 0x7f4a78161df0 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59014/0 (socket says 172.21.15.148:59014) 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 -- 172.21.15.148:0/506453762 learned_addr learned my addr 172.21.15.148:0/506453762 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a77fff640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 0x7f4a78157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7d5a9640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 0x7f4a781f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 -- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 msgr2=0x7f4a781f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 0x7f4a781f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 -- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 msgr2=0x7f4a78157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 0x7f4a78157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7cda8640 1 -- 172.21.15.148:0/506453762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780c54c0 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a77fff640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 0x7f4a78157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.143+0000 7f4a7d5a9640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 0x7f4a781f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a7cda8640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00ec10 tx=0x7f4a6c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c017070 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00cc30 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a780c57a0 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c013470 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.144+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a780c5d60 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.145+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78074f60 con 0x7f4a780cf660 2024-03-04T12:12:48.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.145+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4a6c01a070 con 0x7f4a780cf660 2024-03-04T12:12:48.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.146+0000 7f4a757fa640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 0x7f4a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.146+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4a6c08a8f0 con 0x7f4a780cf660 2024-03-04T12:12:48.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.146+0000 7f4a77fff640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 0x7f4a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.146+0000 7f4a77fff640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 0x7f4a5c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a64007fd0 tx=0x7f4a64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.150+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a6c059370 con 0x7f4a780cf660 2024-03-04T12:12:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.322+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a7815b890 con 0x7f4a780cf660 2024-03-04T12:12:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.323+0000 7f4a757fa640 1 -- 172.21.15.148:0/506453762 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a6c058c70 con 0x7f4a780cf660 2024-03-04T12:12:48.324 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:48.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.324+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 msgr2=0x7f4a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.324+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 0x7f4a5c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a64007fd0 tx=0x7f4a64009040 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 msgr2=0x7f4a78157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00ec10 tx=0x7f4a6c00c520 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 shutdown_connections 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f4a5c06b4d0 0x7f4a5c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a78161df0 0x7f4a781f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a7815da20 0x7f4a78157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 --2- 172.21.15.148:0/506453762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a780cf660 0x7f4a78157d10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 >> 172.21.15.148:0/506453762 conn(0x7f4a780b33a0 msgr2=0x7f4a780b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 shutdown_connections 2024-03-04T12:12:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.325+0000 7f4a7f00c640 1 -- 172.21.15.148:0/506453762 wait complete. 2024-03-04T12:12:48.337 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:48.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.440+0000 7f7aaa69a640 1 Processor -- start 2024-03-04T12:12:48.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.440+0000 7f7aaa69a640 1 -- start start 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aaa69a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 0x7f7aa40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aaa69a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7aa4161df0 0x7f7aa40cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aaa69a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa4162450 con 0x7f7aa40cf660 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aaa69a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40d0000 con 0x7f7aa4161df0 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aaa69a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40cc920 con 0x7f7aa415da20 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa8c37640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7aa4161df0 0x7f7aa40cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7aa40cf660 0x7f7aa41616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50226/0 (socket says 172.21.15.148:50226) 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa37fe640 1 -- 172.21.15.148:0/1557063580 learned_addr learned my addr 172.21.15.148:0/1557063580 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa8c37640 1 -- 172.21.15.148:0/1557063580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 msgr2=0x7f7aa40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa3fff640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 0x7f7aa40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa2ffd640 1 -- 172.21.15.148:0/1557063580 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718229679 0 0) 0x7f7aa4162450 con 0x7f7aa40cf660 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa8c37640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 0x7f7aa40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa8c37640 1 -- 172.21.15.148:0/1557063580 >> v1:172.21.15.148:6789/0 conn(0x7f7aa40cf660 legacy=0x7f7aa41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa8c37640 1 -- 172.21.15.148:0/1557063580 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40cc2d0 con 0x7f7aa4161df0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.441+0000 7f7aa3fff640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 0x7f7aa40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aa8c37640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa40cfac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d90 tx=0x7f7a9c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a432b5d269b5ba51 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aa2ffd640 1 -- 172.21.15.148:0/1557063580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c039070 con 0x7f7aa4161df0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aa2ffd640 1 -- 172.21.15.148:0/1557063580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c02cc50 con 0x7f7aa4161df0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aa2ffd640 1 -- 172.21.15.148:0/1557063580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c034440 con 0x7f7aa4161df0 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aaa69a640 1 -- 172.21.15.148:0/1557063580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 msgr2=0x7f7aa40cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa40cfac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d90 tx=0x7f7a9c02bea0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aaa69a640 1 -- 172.21.15.148:0/1557063580 shutdown_connections 2024-03-04T12:12:48.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa40cfac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.442+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/1557063580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa415da20 0x7f7aa40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.443+0000 7f7aaa69a640 1 -- 172.21.15.148:0/1557063580 >> 172.21.15.148:0/1557063580 conn(0x7f7aa40b33a0 msgr2=0x7f7aa40b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.443+0000 7f7aaa69a640 1 -- 172.21.15.148:0/1557063580 shutdown_connections 2024-03-04T12:12:48.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.443+0000 7f7aaa69a640 1 -- 172.21.15.148:0/1557063580 wait complete. 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.444+0000 7f7aaa69a640 1 Processor -- start 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.444+0000 7f7aaa69a640 1 -- start start 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa40cf660 0x7f7aa40b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 0x7f7aa415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40cbf80 con 0x7f7aa4161df0 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40cd2f0 con 0x7f7aa415da20 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa415de00 con 0x7f7aa40cf660 2024-03-04T12:12:48.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59028/0 (socket says 172.21.15.148:59028) 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 -- 172.21.15.148:0/780915596 learned_addr learned my addr 172.21.15.148:0/780915596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa3fff640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa40cf660 0x7f7aa40b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa37fe640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 0x7f7aa415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 -- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa40cf660 msgr2=0x7f7aa40b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa40cf660 0x7f7aa40b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 -- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 msgr2=0x7f7aa415f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 0x7f7aa415f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 -- 172.21.15.148:0/780915596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa415f8e0 con 0x7f7aa4161df0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa37fe640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 0x7f7aa415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa8c37640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009eb0 tx=0x7f7a9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c039050 con 0x7f7aa4161df0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c045070 con 0x7f7aa4161df0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa415fbc0 con 0x7f7aa4161df0 2024-03-04T12:12:48.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c03e610 con 0x7f7aa4161df0 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.445+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40b4610 con 0x7f7aa4161df0 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.446+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4074f60 con 0x7f7aa4161df0 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.447+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7a9c03e7b0 con 0x7f7aa4161df0 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.447+0000 7f7aa0ff9640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.447+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7a9c0ad0f0 con 0x7f7aa4161df0 2024-03-04T12:12:48.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.447+0000 7f7aa3fff640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.448+0000 7f7aa3fff640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 0x7f7a9006d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40c8600 tx=0x7f7a94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.451+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a9c032e20 con 0x7f7aa4161df0 2024-03-04T12:12:48.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.620+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7aa40d1c40 con 0x7f7aa4161df0 2024-03-04T12:12:48.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.621+0000 7f7aa0ff9640 1 -- 172.21.15.148:0/780915596 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a9c07bc00 con 0x7f7aa4161df0 2024-03-04T12:12:48.623 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:48.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 msgr2=0x7f7a9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 0x7f7a9006d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40c8600 tx=0x7f7a94008040 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 msgr2=0x7f7aa415acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009eb0 tx=0x7f7a9c004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 shutdown_connections 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa4161df0 0x7f7aa415acb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa415da20 0x7f7aa415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 --2- 172.21.15.148:0/780915596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa40cf660 0x7f7aa40b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 >> 172.21.15.148:0/780915596 conn(0x7f7aa40b33a0 msgr2=0x7f7aa40b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 shutdown_connections 2024-03-04T12:12:48.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.623+0000 7f7aaa69a640 1 -- 172.21.15.148:0/780915596 wait complete. 2024-03-04T12:12:48.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:48.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.738+0000 7f6cbd4d1640 1 Processor -- start 2024-03-04T12:12:48.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.738+0000 7f6cbd4d1640 1 -- start start 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cbd4d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 0x7f6cb80d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cbd4d1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6cb80d34a0 0x7f6cb806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cbd4d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb80d2400 con 0x7f6cb80b9410 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cbd4d1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb80d26e0 con 0x7f6cb80d34a0 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cbd4d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb80d29c0 con 0x7f6cb80b5700 2024-03-04T12:12:48.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6cb80b9410 0x7f6cb806c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50230/0 (socket says 172.21.15.148:50230) 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb77fe640 1 -- 172.21.15.148:0/686347357 learned_addr learned my addr 172.21.15.148:0/686347357 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/686347357 >> v2:172.21.15.191:3300/0 conn(0x7f6cb80d34a0 0x7f6cb806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 0x7f6cb80d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb5ffb640 1 -- 172.21.15.148:0/686347357 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035371394 0 0) 0x7f6cb80d2400 con 0x7f6cb80b9410 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb5ffb640 1 -- 172.21.15.148:0/686347357 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca0003520 con 0x7f6cb80b9410 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb67fc640 1 -- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 msgr2=0x7f6cb80d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 0x7f6cb80d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb67fc640 1 -- 172.21.15.148:0/686347357 >> v1:172.21.15.148:6789/0 conn(0x7f6cb80b9410 legacy=0x7f6cb806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb67fc640 1 -- 172.21.15.148:0/686347357 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80d2ca0 con 0x7f6cb80d34a0 2024-03-04T12:12:48.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.739+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 0x7f6cb80d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.740+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80d34a0 0x7f6cb806c0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0097a0 tx=0x7f6cac02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76112454e68dc0ec server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.740+0000 7f6cb5ffb640 1 -- 172.21.15.148:0/686347357 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0315a0 con 0x7f6cb80d34a0 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.740+0000 7f6cb5ffb640 1 -- 172.21.15.148:0/686347357 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac037070 con 0x7f6cb80d34a0 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.740+0000 7f6cb5ffb640 1 -- 172.21.15.148:0/686347357 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac031940 con 0x7f6cb80d34a0 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80d34a0 msgr2=0x7f6cb806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80d34a0 0x7f6cb806c0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0097a0 tx=0x7f6cac02b840 comp rx=0 tx=0).stop 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/686347357 shutdown_connections 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80d34a0 0x7f6cb806c0e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/686347357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80b5700 0x7f6cb80d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/686347357 >> 172.21.15.148:0/686347357 conn(0x7f6cb80d0490 msgr2=0x7f6cb80d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/686347357 shutdown_connections 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.741+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/686347357 wait complete. 2024-03-04T12:12:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.742+0000 7f6cbd4d1640 1 Processor -- start 2024-03-04T12:12:48.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.742+0000 7f6cbd4d1640 1 -- start start 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 0x7f6cb80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 0x7f6cb815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb806ceb0 con 0x7f6cb80b9410 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb806cd30 con 0x7f6cb80b5700 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cbd4d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb80b5ae0 con 0x7f6cb80d34a0 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59030/0 (socket says 172.21.15.148:59030) 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 -- 172.21.15.148:0/671053966 learned_addr learned my addr 172.21.15.148:0/671053966 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 0x7f6cb80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb77fe640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 0x7f6cb815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 -- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 msgr2=0x7f6cb815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 0x7f6cb815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 -- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 msgr2=0x7f6cb80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 0x7f6cb80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb67fc640 1 -- 172.21.15.148:0/671053966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80cc300 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 0x7f6cb80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.743+0000 7f6cb77fe640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 0x7f6cb815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6cb67fc640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6cac02b880 tx=0x7f6cac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac032630 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac0327f0 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb80cc5e0 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac032be0 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.744+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb8157c30 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.745+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb8074f60 con 0x7f6cb80b9410 2024-03-04T12:12:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.745+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6cac04d3f0 con 0x7f6cb80b9410 2024-03-04T12:12:48.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.746+0000 7f6c9f7fe640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:48.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.746+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6cac02f030 con 0x7f6cb80b9410 2024-03-04T12:12:48.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.746+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:48.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.746+0000 7f6cb6ffd640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 0x7f6ca406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ba0 tx=0x7f6ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:48.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.750+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cac040070 con 0x7f6cb80b9410 2024-03-04T12:12:48.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.920+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cb80cd6d0 con 0x7f6cb80b9410 2024-03-04T12:12:48.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.920+0000 7f6c9f7fe640 1 -- 172.21.15.148:0/671053966 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6cac07a5c0 con 0x7f6cb80b9410 2024-03-04T12:12:48.922 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 msgr2=0x7f6ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 0x7f6ca406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ba0 tx=0x7f6ca8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 msgr2=0x7f6cb80c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6cac02b880 tx=0x7f6cac0046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 shutdown_connections 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cb80d34a0 0x7f6cb815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb80b9410 0x7f6cb80c83a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 --2- 172.21.15.148:0/671053966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb80b5700 0x7f6cb80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:48.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 >> 172.21.15.148:0/671053966 conn(0x7f6cb80d0490 msgr2=0x7f6cb80b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:48.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 shutdown_connections 2024-03-04T12:12:48.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:48.922+0000 7f6cbd4d1640 1 -- 172.21.15.148:0/671053966 wait complete. 2024-03-04T12:12:48.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:49.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.036+0000 7fedb3492640 1 Processor -- start 2024-03-04T12:12:49.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.036+0000 7fedb3492640 1 -- start start 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb3492640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cdd70 0x7fedac0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb3492640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fedac0ccfe0 0x7fedac0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb3492640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac06ae00 con 0x7fedac0cc900 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb3492640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac0ccde0 con 0x7fedac0ccfe0 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb3492640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0d0480 con 0x7fedac0cdd70 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fedac0ccfe0 0x7fedac0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb0a2d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fedac0cc900 0x7fedac15d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50242/0 (socket says 172.21.15.148:50242) 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fedac0ccfe0 0x7fedac0cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59036/0 (socket says 172.21.15.148:59036) 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb0a2d640 1 -- 172.21.15.148:0/2307147614 learned_addr learned my addr 172.21.15.148:0/2307147614 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb122e640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cdd70 0x7fedac0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 -- 172.21.15.148:0/2307147614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cdd70 msgr2=0x7fedac0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:49.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cdd70 0x7fedac0cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 -- 172.21.15.148:0/2307147614 >> v1:172.21.15.148:6789/0 conn(0x7fedac0cc900 legacy=0x7fedac15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.037+0000 7fedb1a2f640 1 -- 172.21.15.148:0/2307147614 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac06b0e0 con 0x7fedac0ccfe0 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.038+0000 7fedb1a2f640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0ccfe0 0x7fedac0cff40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feda4009d90 tx=0x7feda402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c37934a78aa377 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.038+0000 7feda3fff640 1 -- 172.21.15.148:0/2307147614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4039070 con 0x7fedac0ccfe0 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.038+0000 7feda3fff640 1 -- 172.21.15.148:0/2307147614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda402cbc0 con 0x7fedac0ccfe0 2024-03-04T12:12:49.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.038+0000 7feda3fff640 1 -- 172.21.15.148:0/2307147614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4034440 con 0x7fedac0ccfe0 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 -- 172.21.15.148:0/2307147614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0ccfe0 msgr2=0x7fedac0cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0ccfe0 0x7fedac0cff40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feda4009d90 tx=0x7feda402be10 comp rx=0 tx=0).stop 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 -- 172.21.15.148:0/2307147614 shutdown_connections 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0ccfe0 0x7fedac0cff40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 --2- 172.21.15.148:0/2307147614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cdd70 0x7fedac0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 -- 172.21.15.148:0/2307147614 >> 172.21.15.148:0/2307147614 conn(0x7fedac0b22e0 msgr2=0x7fedac0b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 -- 172.21.15.148:0/2307147614 shutdown_connections 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.039+0000 7fedb3492640 1 -- 172.21.15.148:0/2307147614 wait complete. 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.040+0000 7fedb3492640 1 Processor -- start 2024-03-04T12:12:49.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.040+0000 7fedb3492640 1 -- start start 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 0x7fedac0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 0x7fedac0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0b2e90 con 0x7fedac0cdd70 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac06b6a0 con 0x7fedac0ccfe0 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb3492640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0b3010 con 0x7fedac0cc900 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59042/0 (socket says 172.21.15.148:59042) 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 -- 172.21.15.148:0/4163401503 learned_addr learned my addr 172.21.15.148:0/4163401503 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:49.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb122e640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 0x7fedac0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb0a2d640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 0x7fedac0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 -- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 msgr2=0x7fedac0ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 0x7fedac0ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 -- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 msgr2=0x7fedac0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 0x7fedac0b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 -- 172.21.15.148:0/4163401503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0c4a00 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb0a2d640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 0x7fedac0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb122e640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 0x7fedac0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.041+0000 7fedb1a2f640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feda4009eb0 tx=0x7feda4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.042+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4039050 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.042+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda4045070 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.042+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda403e830 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.042+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac1e7640 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.042+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1e7c00 con 0x7fedac0cdd70 2024-03-04T12:12:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.043+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0cea60 con 0x7fedac0cdd70 2024-03-04T12:12:49.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.043+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feda403e9d0 con 0x7fedac0cdd70 2024-03-04T12:12:49.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.043+0000 7feda1ffb640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 0x7fed9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.044+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feda40ad380 con 0x7fedac0cdd70 2024-03-04T12:12:49.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.044+0000 7fedb122e640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 0x7fed9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.044+0000 7fedb122e640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 0x7fed9006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feda8006fd0 tx=0x7feda8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.048+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedac0cea60 con 0x7fedac0cdd70 2024-03-04T12:12:49.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.219+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fedac074f60 con 0x7fedac0cdd70 2024-03-04T12:12:49.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.219+0000 7feda1ffb640 1 -- 172.21.15.148:0/4163401503 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7feda4032e20 con 0x7fedac0cdd70 2024-03-04T12:12:49.221 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 msgr2=0x7fed9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 0x7fed9006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feda8006fd0 tx=0x7feda8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 msgr2=0x7fedac0b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feda4009eb0 tx=0x7feda4004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:49.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 shutdown_connections 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fed9006b4d0 0x7fed9006d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedac0cdd70 0x7fedac0b32d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedac0ccfe0 0x7fedac0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 --2- 172.21.15.148:0/4163401503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fedac0cc900 0x7fedac0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 >> 172.21.15.148:0/4163401503 conn(0x7fedac0b22e0 msgr2=0x7fedac06e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 shutdown_connections 2024-03-04T12:12:49.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.221+0000 7fedb3492640 1 -- 172.21.15.148:0/4163401503 wait complete. 2024-03-04T12:12:49.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.335+0000 7fce9095b640 1 Processor -- start 2024-03-04T12:12:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.335+0000 7fce9095b640 1 -- start start 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce8c15b560 0x7fce8c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 0x7fce8c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce8c0b3720 0x7fce8c0b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c0b40d0 con 0x7fce8c15b560 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c06d860 con 0x7fce8c0b3720 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce9095b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15ed90 con 0x7fce8c15b940 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce8c15b560 0x7fce8c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce8c15b560 0x7fce8c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59046/0 (socket says 172.21.15.148:59046) 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 -- 172.21.15.148:0/2627560241 learned_addr learned my addr 172.21.15.148:0/2627560241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce83fff640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 0x7fce8c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 -- 172.21.15.148:0/2627560241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 msgr2=0x7fce8c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 0x7fce8c15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 -- 172.21.15.148:0/2627560241 >> v2:172.21.15.191:3300/0 conn(0x7fce8c0b3720 msgr2=0x7fce8c0b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2627560241 >> v2:172.21.15.191:3300/0 conn(0x7fce8c0b3720 0x7fce8c0b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce8a59c640 1 -- 172.21.15.148:0/2627560241 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c15f070 con 0x7fce8c15b560 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.336+0000 7fce83fff640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 0x7fce8c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.337+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b560 0x7fce8c15e260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b2f50 tx=0x7fce7c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266673e606ecdd2c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.337+0000 7fce89d9b640 1 -- 172.21.15.148:0/2627560241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c035020 con 0x7fce8c15b560 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.337+0000 7fce89d9b640 1 -- 172.21.15.148:0/2627560241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c02d040 con 0x7fce8c15b560 2024-03-04T12:12:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.337+0000 7fce89d9b640 1 -- 172.21.15.148:0/2627560241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c031a80 con 0x7fce8c15b560 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 -- 172.21.15.148:0/2627560241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b560 msgr2=0x7fce8c15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b560 0x7fce8c15e260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b2f50 tx=0x7fce7c02b900 comp rx=0 tx=0).stop 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 -- 172.21.15.148:0/2627560241 shutdown_connections 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b940 0x7fce8c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 --2- 172.21.15.148:0/2627560241 >> v2:172.21.15.191:3300/0 conn(0x7fce8c0b3720 0x7fce8c0b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 --2- 172.21.15.148:0/2627560241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b560 0x7fce8c15e260 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 -- 172.21.15.148:0/2627560241 >> 172.21.15.148:0/2627560241 conn(0x7fce8c0cfb40 msgr2=0x7fce8c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 -- 172.21.15.148:0/2627560241 shutdown_connections 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.338+0000 7fce9095b640 1 -- 172.21.15.148:0/2627560241 wait complete. 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.339+0000 7fce9095b640 1 Processor -- start 2024-03-04T12:12:49.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.339+0000 7fce9095b640 1 -- start start 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 0x7fce8c148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b560 0x7fce8c160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c071a90 con 0x7fce8c15b940 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c06de00 con 0x7fce8c0b3720 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c161850 con 0x7fce8c15b560 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59060/0 (socket says 172.21.15.148:59060) 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 -- 172.21.15.148:0/2733493827 learned_addr learned my addr 172.21.15.148:0/2733493827 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce83fff640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b560 0x7fce8c160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 0x7fce8c148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 -- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b560 msgr2=0x7fce8c160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b560 0x7fce8c160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 -- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 msgr2=0x7fce8c148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 0x7fce8c148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 -- 172.21.15.148:0/2733493827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c1f77b0 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 0x7fce8c148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce8ad9d640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b970 tx=0x7fce8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84019070 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8400cb60 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c1f7a90 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84014440 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.340+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c0c4fb0 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.341+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c074f60 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.342+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fce8401c070 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.342+0000 7fce82ffd640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 0x7fce6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.342+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fce8408a340 con 0x7fce8c15b940 2024-03-04T12:12:49.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.342+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 0x7fce6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.343+0000 7fce8a59c640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 0x7fce6c06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce7c036000 tx=0x7fce7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.346+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce84058dc0 con 0x7fce8c15b940 2024-03-04T12:12:49.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.530+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce8c0cf920 con 0x7fce8c15b940 2024-03-04T12:12:49.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.530+0000 7fce82ffd640 1 -- 172.21.15.148:0/2733493827 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce840586c0 con 0x7fce8c15b940 2024-03-04T12:12:49.532 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 msgr2=0x7fce6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 0x7fce6c06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce7c036000 tx=0x7fce7c008030 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 msgr2=0x7fce8c1f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b970 tx=0x7fce8400be30 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 shutdown_connections 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fce6c06b4d0 0x7fce6c06d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce8c15b940 0x7fce8c1f6cf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce8c15b560 0x7fce8c160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 --2- 172.21.15.148:0/2733493827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8c0b3720 0x7fce8c148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 >> 172.21.15.148:0/2733493827 conn(0x7fce8c0cfb40 msgr2=0x7fce8c0b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 shutdown_connections 2024-03-04T12:12:49.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.532+0000 7fce9095b640 1 -- 172.21.15.148:0/2733493827 wait complete. 2024-03-04T12:12:49.545 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-04T12:12:49.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:49.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-04T12:12:49.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T12:12:49.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:49.574 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-04T12:12:49.629 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:49.629 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:49.649 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:49.649 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:49.707 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T12:12:49.707 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:12:49.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:49.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.865+0000 7fd414316640 1 Processor -- start 2024-03-04T12:12:49.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.865+0000 7fd414316640 1 -- start start 2024-03-04T12:12:49.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd414316640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd40c15b030 0x7fd40c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd414316640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c06d3b0 0x7fd40c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd414316640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c15e2d0 con 0x7fd40c15b410 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd414316640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0b2720 con 0x7fd40c15b030 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd414316640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06d8b0 con 0x7fd40c06d3b0 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4120b2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd40c15b030 0x7fd40c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4128b3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd40c15b410 0x7fd40c15b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50244/0 (socket says 172.21.15.148:50244) 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4128b3640 1 -- 172.21.15.148:0/114164165 learned_addr learned my addr 172.21.15.148:0/114164165 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4118b1640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c06d3b0 0x7fd40c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4120b2640 1 -- 172.21.15.148:0/114164165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c06d3b0 msgr2=0x7fd40c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4120b2640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c06d3b0 0x7fd40c15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4120b2640 1 -- 172.21.15.148:0/114164165 >> v1:172.21.15.148:6789/0 conn(0x7fd40c15b410 legacy=0x7fd40c15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4120b2640 1 -- 172.21.15.148:0/114164165 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c06db90 con 0x7fd40c15b030 2024-03-04T12:12:49.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.866+0000 7fd4110b0640 1 -- 172.21.15.148:0/114164165 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238979554 0 0) 0x7fd40c15e2d0 con 0x7fd40c15b410 2024-03-04T12:12:49.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd4120b2640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c15c120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd4080097a0 tx=0x7fd40802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe22b4c7388e2efc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd4110b0640 1 -- 172.21.15.148:0/114164165 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080315a0 con 0x7fd40c15b030 2024-03-04T12:12:49.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd4110b0640 1 -- 172.21.15.148:0/114164165 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408037070 con 0x7fd40c15b030 2024-03-04T12:12:49.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd4110b0640 1 -- 172.21.15.148:0/114164165 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408031960 con 0x7fd40c15b030 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd414316640 1 -- 172.21.15.148:0/114164165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 msgr2=0x7fd40c15c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.867+0000 7fd414316640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c15c120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd4080097a0 tx=0x7fd40802b840 comp rx=0 tx=0).stop 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 -- 172.21.15.148:0/114164165 shutdown_connections 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c06d3b0 0x7fd40c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 --2- 172.21.15.148:0/114164165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c15c120 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 -- 172.21.15.148:0/114164165 >> 172.21.15.148:0/114164165 conn(0x7fd40c06acd0 msgr2=0x7fd40c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 -- 172.21.15.148:0/114164165 shutdown_connections 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.868+0000 7fd414316640 1 -- 172.21.15.148:0/114164165 wait complete. 2024-03-04T12:12:49.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 Processor -- start 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 -- start start 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd40c06d3b0 0x7fd40c0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 0x7fd40c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06e690 con 0x7fd40c15b030 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06e150 con 0x7fd40c06d3b0 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd414316640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06e810 con 0x7fd40c15b410 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.869+0000 7fd4118b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59068/0 (socket says 172.21.15.148:59068) 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 -- 172.21.15.148:0/1836276225 learned_addr learned my addr 172.21.15.148:0/1836276225 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4128b3640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 0x7fd40c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4120b2640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd40c06d3b0 0x7fd40c0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 -- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 msgr2=0x7fd40c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 0x7fd40c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 -- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd40c06d3b0 msgr2=0x7fd40c0ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd40c06d3b0 0x7fd40c0ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 -- 172.21.15.148:0/1836276225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0a9650 con 0x7fd40c15b030 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4128b3640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 0x7fd40c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd4118b1640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00d5f0 tx=0x7fd3fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0125f0 con 0x7fd40c15b030 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc019070 con 0x7fd40c15b030 2024-03-04T12:12:49.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c0a9930 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0129d0 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.870+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0a9ef0 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.871+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c074f60 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.872+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd3fc012b70 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.872+0000 7fd402ffd640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 0x7fd3f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.872+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd3fc08aa10 con 0x7fd40c15b030 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.872+0000 7fd4120b2640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 0x7fd3f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:49.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.872+0000 7fd4120b2640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 0x7fd3f006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd408036000 tx=0x7fd40803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:49.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:49.876+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3fc059490 con 0x7fd40c15b030 2024-03-04T12:12:50.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.050+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40c15a3a0 con 0x7fd40c15b030 2024-03-04T12:12:50.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.051+0000 7fd402ffd640 1 -- 172.21.15.148:0/1836276225 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fd3fc058d90 con 0x7fd40c15b030 2024-03-04T12:12:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:50.053 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 msgr2=0x7fd3f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 0x7fd3f006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd408036000 tx=0x7fd40803d040 comp rx=0 tx=0).stop 2024-03-04T12:12:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 msgr2=0x7fd40c155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00d5f0 tx=0x7fd3fc00dab0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 shutdown_connections 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd3f006b4d0 0x7fd3f006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd40c15b410 0x7fd40c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c15b030 0x7fd40c155390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 --2- 172.21.15.148:0/1836276225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd40c06d3b0 0x7fd40c0ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 >> 172.21.15.148:0/1836276225 conn(0x7fd40c06acd0 msgr2=0x7fd40c0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 shutdown_connections 2024-03-04T12:12:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.053+0000 7fd414316640 1 -- 172.21.15.148:0/1836276225 wait complete. 2024-03-04T12:12:50.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:50.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:50.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.168+0000 7fbb08c80640 1 Processor -- start 2024-03-04T12:12:50.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.168+0000 7fbb08c80640 1 -- start start 2024-03-04T12:12:50.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.169+0000 7fbb08c80640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 0x7fbb040b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.169+0000 7fbb08c80640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.169+0000 7fbb08c80640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0406d8a0 con 0x7fbb040cbfb0 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.169+0000 7fbb08c80640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0406db80 con 0x7fbb0415a810 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.169+0000 7fbb08c80640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0406de60 con 0x7fbb0406d3c0 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb0259c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 0x7fbb040b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:33686/0 (socket says 172.21.15.148:33686) 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 -- 172.21.15.148:0/3903121273 learned_addr learned my addr 172.21.15.148:0/3903121273 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 -- 172.21.15.148:0/3903121273 >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 msgr2=0x7fbb040b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/3903121273 >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 0x7fbb040b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 -- 172.21.15.148:0/3903121273 >> v1:172.21.15.191:6789/0 conn(0x7fbb040cbfb0 legacy=0x7fbb0415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb01d9b640 1 -- 172.21.15.148:0/3903121273 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0406e140 con 0x7fbb0406d3c0 2024-03-04T12:12:50.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.170+0000 7fbb0259c640 1 --2- 172.21.15.148:0/3903121273 >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 0x7fbb040b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.171+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/3903121273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800bb10 tx=0x7fbaf802e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed29c0716b8f7ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.171+0000 7fbb0159a640 1 -- 172.21.15.148:0/3903121273 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf803b070 con 0x7fbb0406d3c0 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.171+0000 7fbb0159a640 1 -- 172.21.15.148:0/3903121273 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf8036440 con 0x7fbb0406d3c0 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.171+0000 7fbb0159a640 1 -- 172.21.15.148:0/3903121273 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8036810 con 0x7fbb0406d3c0 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 -- 172.21.15.148:0/3903121273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 msgr2=0x7fbb040b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 --2- 172.21.15.148:0/3903121273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800bb10 tx=0x7fbaf802e150 comp rx=0 tx=0).stop 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 -- 172.21.15.148:0/3903121273 shutdown_connections 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 --2- 172.21.15.148:0/3903121273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0406d3c0 0x7fbb040b46d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 --2- 172.21.15.148:0/3903121273 >> v2:172.21.15.191:3300/0 conn(0x7fbb0415a810 0x7fbb040b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 -- 172.21.15.148:0/3903121273 >> 172.21.15.148:0/3903121273 conn(0x7fbb0406acd0 msgr2=0x7fbb0406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 -- 172.21.15.148:0/3903121273 shutdown_connections 2024-03-04T12:12:50.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.172+0000 7fbb08c80640 1 -- 172.21.15.148:0/3903121273 wait complete. 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.173+0000 7fbb08c80640 1 Processor -- start 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.173+0000 7fbb08c80640 1 -- start start 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 0x7fbb04154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0415a810 0x7fbb04154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0415e3c0 con 0x7fbb0406d3c0 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb040b3c10 con 0x7fbb040cbfb0 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0415e540 con 0x7fbb0415a810 2024-03-04T12:12:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59082/0 (socket says 172.21.15.148:59082) 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 -- 172.21.15.148:0/2161679955 learned_addr learned my addr 172.21.15.148:0/2161679955 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 0x7fbb04154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb02d9d640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0415a810 0x7fbb04154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 -- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0415a810 msgr2=0x7fbb04154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0415a810 0x7fbb04154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 -- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 msgr2=0x7fbb04154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 0x7fbb04154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 -- 172.21.15.148:0/2161679955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0415bb20 con 0x7fbb0406d3c0 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 0x7fbb04154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb0259c640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400efc0 tx=0x7fbaf400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4018020 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4014680 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4014a50 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.174+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0415be00 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.175+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0415c3c0 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.175+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb04074f60 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.176+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbaf4005d10 con 0x7fbb0406d3c0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.176+0000 7fbaf2ffd640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 0x7fbae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.176+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 0x7fbae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.177+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbaf408b3d0 con 0x7fbb0406d3c0 2024-03-04T12:12:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.177+0000 7fbb01d9b640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 0x7fbae406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8038000 tx=0x7fbaf8032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.181+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaf4022030 con 0x7fbb0406d3c0 2024-03-04T12:12:50.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.353+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb0415c5c0 con 0x7fbb0406d3c0 2024-03-04T12:12:50.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.354+0000 7fbaf2ffd640 1 -- 172.21.15.148:0/2161679955 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbaf4010040 con 0x7fbb0406d3c0 2024-03-04T12:12:50.356 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 msgr2=0x7fbae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 0x7fbae406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8038000 tx=0x7fbaf8032000 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 msgr2=0x7fbb04152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400efc0 tx=0x7fbaf400c490 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 shutdown_connections 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbae406b4d0 0x7fbae406d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb0415a810 0x7fbb04154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb040cbfb0 0x7fbb04154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 --2- 172.21.15.148:0/2161679955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb0406d3c0 0x7fbb04152690 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 >> 172.21.15.148:0/2161679955 conn(0x7fbb0406acd0 msgr2=0x7fbb040afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 shutdown_connections 2024-03-04T12:12:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.356+0000 7fbb08c80640 1 -- 172.21.15.148:0/2161679955 wait complete. 2024-03-04T12:12:50.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:50.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.471+0000 7f0b7bc59640 1 Processor -- start 2024-03-04T12:12:50.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.471+0000 7f0b7bc59640 1 -- start start 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7bc59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b740b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7bc59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b740d1e90 0x7f0b740cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7bc59640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7415bf50 con 0x7f0b7415f660 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7bc59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b740cf890 con 0x7f0b740d1e90 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7bc59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7415fb60 con 0x7f0b7415bb70 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b740d1e90 0x7f0b740cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b740d1e90 0x7f0b740cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59088/0 (socket says 172.21.15.148:59088) 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 -- 172.21.15.148:0/2310269258 learned_addr learned my addr 172.21.15.148:0/2310269258 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b799f5640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b740b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 -- 172.21.15.148:0/2310269258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 msgr2=0x7f0b740b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b740b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 -- 172.21.15.148:0/2310269258 >> v1:172.21.15.191:6789/0 conn(0x7f0b7415f660 legacy=0x7f0b740d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.472+0000 7f0b7a1f6640 1 -- 172.21.15.148:0/2310269258 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7415fe40 con 0x7f0b740d1e90 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.473+0000 7f0b7a1f6640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b740cfd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009e50 tx=0x7f0b6c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe62dd4c203e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.473+0000 7f0b789f3640 1 -- 172.21.15.148:0/2310269258 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c02c9e0 con 0x7f0b740d1e90 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.473+0000 7f0b789f3640 1 -- 172.21.15.148:0/2310269258 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c02cb80 con 0x7f0b740d1e90 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.473+0000 7f0b789f3640 1 -- 172.21.15.148:0/2310269258 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0343f0 con 0x7f0b740d1e90 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.473+0000 7f0b7bc59640 1 -- 172.21.15.148:0/2310269258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 msgr2=0x7f0b740cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b740cfd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009e50 tx=0x7f0b6c02be40 comp rx=0 tx=0).stop 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 -- 172.21.15.148:0/2310269258 shutdown_connections 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b740cfd20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/2310269258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b740b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 -- 172.21.15.148:0/2310269258 >> 172.21.15.148:0/2310269258 conn(0x7f0b740b33a0 msgr2=0x7f0b740b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 -- 172.21.15.148:0/2310269258 shutdown_connections 2024-03-04T12:12:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.474+0000 7f0b7bc59640 1 -- 172.21.15.148:0/2310269258 wait complete. 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 Processor -- start 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 -- start start 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b741f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 0x7f0b741f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740b0f80 con 0x7f0b740d1e90 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74160400 con 0x7f0b7415f660 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.475+0000 7f0b7bc59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740b1100 con 0x7f0b7415bb70 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59098/0 (socket says 172.21.15.148:59098) 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 -- 172.21.15.148:0/3825858908 learned_addr learned my addr 172.21.15.148:0/3825858908 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b7a1f6640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 0x7f0b741f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b791f4640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b741f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 -- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 msgr2=0x7f0b741f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b741f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 -- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 msgr2=0x7f0b741f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 0x7f0b741f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 -- 172.21.15.148:0/3825858908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b741e5ef0 con 0x7f0b740d1e90 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b7a1f6640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 0x7f0b741f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b791f4640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b741f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b799f5640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000b6c0 con 0x7f0b740d1e90 2024-03-04T12:12:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b70019070 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b741e61d0 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000ba80 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.476+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b741e6a70 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.477+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b741e63a0 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.478+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0b7000bc20 con 0x7f0b740d1e90 2024-03-04T12:12:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.478+0000 7f0b6a7fc640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.478+0000 7f0b791f4640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.478+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0b7008aab0 con 0x7f0b740d1e90 2024-03-04T12:12:50.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.479+0000 7f0b791f4640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 0x7f0b5806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0b741e73c0 tx=0x7f0b64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.483+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b70059530 con 0x7f0b740d1e90 2024-03-04T12:12:50.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.654+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b74074f60 con 0x7f0b740d1e90 2024-03-04T12:12:50.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.655+0000 7f0b6a7fc640 1 -- 172.21.15.148:0/3825858908 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b70058e30 con 0x7f0b740d1e90 2024-03-04T12:12:50.657 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 msgr2=0x7f0b5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 0x7f0b5806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0b741e73c0 tx=0x7f0b64008040 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 msgr2=0x7f0b741e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b7000dcc0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 shutdown_connections 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b7415f660 0x7f0b741f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b7415bb70 0x7f0b741f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 --2- 172.21.15.148:0/3825858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b740d1e90 0x7f0b741e5650 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 >> 172.21.15.148:0/3825858908 conn(0x7f0b740b33a0 msgr2=0x7f0b740b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 shutdown_connections 2024-03-04T12:12:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.657+0000 7f0b7bc59640 1 -- 172.21.15.148:0/3825858908 wait complete. 2024-03-04T12:12:50.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:50.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.772+0000 7fc227f97640 1 Processor -- start 2024-03-04T12:12:50.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.772+0000 7fc227f97640 1 -- start start 2024-03-04T12:12:50.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc227f97640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc22006ff60 0x7fc2200b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc227f97640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc220070340 con 0x7fc2200cf190 2024-03-04T12:12:50.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc227f97640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2200cb440 con 0x7fc2200cd820 2024-03-04T12:12:50.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc227f97640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2200cc050 con 0x7fc22006ff60 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc225532640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc2200cf190 0x7fc2200cd110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50246/0 (socket says 172.21.15.148:50246) 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc225532640 1 -- 172.21.15.148:0/4167418630 learned_addr learned my addr 172.21.15.148:0/4167418630 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc225d33640 1 --2- 172.21.15.148:0/4167418630 >> v2:172.21.15.191:3300/0 conn(0x7fc22006ff60 0x7fc2200b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494058223 0 0) 0x7fc220070340 con 0x7fc2200cf190 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c003520 con 0x7fc2200cf190 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58258194 0 0) 0x7fc2200cb440 con 0x7fc2200cd820 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc220070340 con 0x7fc2200cd820 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 469274576 0 0) 0x7fc20c003520 con 0x7fc2200cf190 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 >> v2:172.21.15.191:3300/0 conn(0x7fc22006ff60 msgr2=0x7fc2200b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 --2- 172.21.15.148:0/4167418630 >> v2:172.21.15.191:3300/0 conn(0x7fc22006ff60 0x7fc2200b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 >> v1:172.21.15.191:6789/0 conn(0x7fc2200cd820 legacy=0x7fc22015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200cb720 con 0x7fc2200cf190 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fc210001670 con 0x7fc2200cf190 2024-03-04T12:12:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 >> v1:172.21.15.148:6789/0 conn(0x7fc2200cf190 legacy=0x7fc2200cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c003b90 0x7fc20c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c0051c0 0x7fc20c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cc050 con 0x7fc20c004800 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00e2a0 con 0x7fc20c003b90 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00e4a0 con 0x7fc20c0051c0 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.773+0000 7fc225532640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225d33640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c003b90 0x7fc20c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc226534640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c0051c0 0x7fc20c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 -- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c0051c0 msgr2=0x7fc20c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c0051c0 0x7fc20c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 -- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c003b90 msgr2=0x7fc20c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c003b90 0x7fc20c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 -- 172.21.15.148:0/4167418630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c003520 con 0x7fc20c004800 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc226534640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c0051c0 0x7fc20c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc226534640 1 -- 172.21.15.148:0/4167418630 reap_dead start 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.774+0000 7fc225532640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c0e0 tx=0x7fc20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb5ef3d3a0777af server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210007c40 con 0x7fc20c004800 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210030070 con 0x7fc20c004800 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc224d31640 1 -- 172.21.15.148:0/4167418630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210005d90 con 0x7fc20c004800 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 -- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 msgr2=0x7fc20c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c0e0 tx=0x7fc20c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 -- 172.21.15.148:0/4167418630 shutdown_connections 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 --2- 172.21.15.148:0/4167418630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 -- 172.21.15.148:0/4167418630 >> 172.21.15.148:0/4167418630 conn(0x7fc2200b2250 msgr2=0x7fc2200b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 -- 172.21.15.148:0/4167418630 shutdown_connections 2024-03-04T12:12:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.775+0000 7fc227f97640 1 -- 172.21.15.148:0/4167418630 wait complete. 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.776+0000 7fc227f97640 1 Processor -- start 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.776+0000 7fc227f97640 1 -- start start 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c004800 0x7fc2200c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2200c7ff0 0x7fc2200c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cc4e0 con 0x7fc2200c8990 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cbce0 con 0x7fc2200c7ff0 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc227f97640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200ae320 con 0x7fc20c004800 2024-03-04T12:12:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc226534640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc226534640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59120/0 (socket says 172.21.15.148:59120) 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc226534640 1 -- 172.21.15.148:0/759128413 learned_addr learned my addr 172.21.15.148:0/759128413 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.777+0000 7fc225d33640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c004800 0x7fc2200c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc225532640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2200c7ff0 0x7fc2200c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 -- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c004800 msgr2=0x7fc2200c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c004800 0x7fc2200c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 -- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2200c7ff0 msgr2=0x7fc2200c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2200c7ff0 0x7fc2200c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 -- 172.21.15.148:0/759128413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200ae600 con 0x7fc2200c8990 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc226534640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc22006d000 tx=0x7fc21800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21800ee50 con 0x7fc2200c8990 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218003b80 con 0x7fc2200c8990 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218003d70 con 0x7fc2200c8990 2024-03-04T12:12:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2200ae8e0 con 0x7fc2200c8990 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.778+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2200aeea0 con 0x7fc2200c8990 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.779+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2200cf930 con 0x7fc2200c8990 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.779+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc21801a020 con 0x7fc2200c8990 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.780+0000 7fc2167fc640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.780+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc218089b40 con 0x7fc2200c8990 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.780+0000 7fc225d33640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.780+0000 7fc225d33640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 0x7fc20406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009b00 tx=0x7fc21c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.784+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2200cf930 con 0x7fc2200c8990 2024-03-04T12:12:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.957+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc220074f60 con 0x7fc2200c8990 2024-03-04T12:12:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.957+0000 7fc2167fc640 1 -- 172.21.15.148:0/759128413 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc218021020 con 0x7fc2200c8990 2024-03-04T12:12:50.959 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:50.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 msgr2=0x7fc20406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 0x7fc20406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009b00 tx=0x7fc21c008040 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 msgr2=0x7fc2200c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc22006d000 tx=0x7fc21800ec00 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 shutdown_connections 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2200c8990 0x7fc2200c4ff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2200c7ff0 0x7fc2200c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 --2- 172.21.15.148:0/759128413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc20c004800 0x7fc2200c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 >> 172.21.15.148:0/759128413 conn(0x7fc2200b2250 msgr2=0x7fc2200af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 shutdown_connections 2024-03-04T12:12:50.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:50.959+0000 7fc227f97640 1 -- 172.21.15.148:0/759128413 wait complete. 2024-03-04T12:12:50.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:51.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.075+0000 7f41dcb89640 1 Processor -- start 2024-03-04T12:12:51.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.075+0000 7f41dcb89640 1 -- start start 2024-03-04T12:12:51.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.076+0000 7f41dcb89640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41d81621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.076+0000 7f41dcb89640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80d3510 con 0x7f41d815dc60 2024-03-04T12:12:51.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.076+0000 7f41dcb89640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80b9920 con 0x7f41d80b9420 2024-03-04T12:12:51.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.076+0000 7f41dcb89640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8162720 con 0x7f41d80b5c90 2024-03-04T12:12:51.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f41d815dc60 0x7f41d80b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50254/0 (socket says 172.21.15.148:50254) 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d659c640 1 -- 172.21.15.148:0/161463624 learned_addr learned my addr 172.21.15.148:0/161463624 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49094310 0 0) 0x7f41d80d3510 con 0x7f41d815dc60 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c0003520 con 0x7f41d815dc60 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41d81621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404354349 0 0) 0x7f41c0003520 con 0x7f41d815dc60 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 msgr2=0x7f41d81621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41d81621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 >> v1:172.21.15.191:6789/0 conn(0x7f41d80b9420 legacy=0x7f41d80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80d1c40 con 0x7f41d815dc60 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f41c80019f0 con 0x7f41d815dc60 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41d81621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 >> v1:172.21.15.148:6789/0 conn(0x7f41d815dc60 legacy=0x7f41d80b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c0004800 0x7f41c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8162720 con 0x7f41c00039e0 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c000bac0 con 0x7f41c0004800 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c000bcc0 con 0x7f41d80b5c90 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d659c640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c0004800 0x7f41c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d6d9d640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d659c640 1 -- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 msgr2=0x7f41c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.077+0000 7f41d659c640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d659c640 1 -- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c0004800 msgr2=0x7f41c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d659c640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c0004800 0x7f41c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d659c640 1 -- 172.21.15.148:0/161463624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c0003520 con 0x7f41c00039e0 2024-03-04T12:12:51.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c0004800 0x7f41c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d6d9d640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d80b5c90 0x7f41c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d659c640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41c0003e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41c800a860 tx=0x7f41c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb6c839f34c536b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800a480 con 0x7f41c00039e0 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8034630 con 0x7f41c00039e0 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.078+0000 7f41d559a640 1 -- 172.21.15.148:0/161463624 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8034a60 con 0x7f41c00039e0 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 -- 172.21.15.148:0/161463624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 msgr2=0x7f41c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 --2- 172.21.15.148:0/161463624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41c0003e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41c800a860 tx=0x7f41c0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41d6d9d640 1 -- 172.21.15.148:0/161463624 reap_dead start 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 -- 172.21.15.148:0/161463624 shutdown_connections 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 -- 172.21.15.148:0/161463624 >> 172.21.15.148:0/161463624 conn(0x7f41d80d0580 msgr2=0x7f41d80d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 -- 172.21.15.148:0/161463624 shutdown_connections 2024-03-04T12:12:51.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.079+0000 7f41dcb89640 1 -- 172.21.15.148:0/161463624 wait complete. 2024-03-04T12:12:51.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.080+0000 7f41dcb89640 1 Processor -- start 2024-03-04T12:12:51.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.080+0000 7f41dcb89640 1 -- start start 2024-03-04T12:12:51.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d8154c20 0x7f41d81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d806c270 0x7f41d806c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80d2290 con 0x7f41c00039e0 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80d19f0 con 0x7f41d8154c20 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41dcb89640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d815e040 con 0x7f41d806c270 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59130/0 (socket says 172.21.15.148:59130) 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41d659c640 1 -- 172.21.15.148:0/3988459299 learned_addr learned my addr 172.21.15.148:0/3988459299 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.081+0000 7f41d659c640 1 -- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d806c270 msgr2=0x7f41d806c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41d659c640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d806c270 0x7f41d806c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41d659c640 1 -- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d8154c20 msgr2=0x7f41d81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41d659c640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d8154c20 0x7f41d81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41d659c640 1 -- 172.21.15.148:0/3988459299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d815e450 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41d659c640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f41c800a860 tx=0x7f41d80b9c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8039040 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8043070 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d815e730 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8004d60 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.082+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d815ecf0 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.083+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d8074f60 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.083+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f41c8034630 con 0x7f41c00039e0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.084+0000 7f41c6ffd640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 0x7f41b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.084+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 0x7f41b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.084+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f41c80ac530 con 0x7f41c00039e0 2024-03-04T12:12:51.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.084+0000 7f41d5d9b640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 0x7f41b806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006d40 tx=0x7f41cc006c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.088+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41c807afb0 con 0x7f41c00039e0 2024-03-04T12:12:51.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.258+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f41d80d17d0 con 0x7f41c00039e0 2024-03-04T12:12:51.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.258+0000 7f41c6ffd640 1 -- 172.21.15.148:0/3988459299 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f41c807a8b0 con 0x7f41c00039e0 2024-03-04T12:12:51.261 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:51.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 msgr2=0x7f41b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 0x7f41b806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006d40 tx=0x7f41cc006c10 comp rx=0 tx=0).stop 2024-03-04T12:12:51.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 msgr2=0x7f41d8147f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f41c800a860 tx=0x7f41d80b9c00 comp rx=0 tx=0).stop 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 shutdown_connections 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f41b806b500 0x7f41b806d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d806c270 0x7f41d806c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d8154c20 0x7f41d81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 --2- 172.21.15.148:0/3988459299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c00039e0 0x7f41d8147f40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 >> 172.21.15.148:0/3988459299 conn(0x7f41d80d0580 msgr2=0x7f41d80d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 shutdown_connections 2024-03-04T12:12:51.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.260+0000 7f41dcb89640 1 -- 172.21.15.148:0/3988459299 wait complete. 2024-03-04T12:12:51.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T12:12:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.373+0000 7f602df7e640 1 Processor -- start 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.373+0000 7f602df7e640 1 -- start start 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f602df7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f602815de00 0x7f60280b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f602df7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60280b5470 0x7f60280b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f602df7e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602815e1e0 con 0x7f60280b9380 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f602df7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60280d2820 con 0x7f60280b5470 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f602df7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280d1c10 con 0x7f602815de00 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f6027fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60280b5470 0x7f60280b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f6027fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60280b5470 0x7f60280b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59138/0 (socket says 172.21.15.148:59138) 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f6027fff640 1 -- 172.21.15.148:0/1740716285 learned_addr learned my addr 172.21.15.148:0/1740716285 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f60277fe640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f602815de00 0x7f60280b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.374+0000 7f6027fff640 1 -- 172.21.15.148:0/1740716285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f602815de00 msgr2=0x7f60280b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f6027fff640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f602815de00 0x7f60280b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f6027fff640 1 -- 172.21.15.148:0/1740716285 >> v1:172.21.15.191:6789/0 conn(0x7f60280b9380 legacy=0x7f60280b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f6027fff640 1 -- 172.21.15.148:0/1740716285 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280d1ef0 con 0x7f60280b5470 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f6027fff640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280b5470 0x7f60280b5900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e50 tx=0x7f602002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72852cec407dbf94 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f60267fc640 1 -- 172.21.15.148:0/1740716285 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602002ca70 con 0x7f60280b5470 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f60267fc640 1 -- 172.21.15.148:0/1740716285 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602002cc10 con 0x7f60280b5470 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.375+0000 7f60267fc640 1 -- 172.21.15.148:0/1740716285 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200343f0 con 0x7f60280b5470 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 -- 172.21.15.148:0/1740716285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280b5470 msgr2=0x7f60280b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280b5470 0x7f60280b5900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e50 tx=0x7f602002bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 -- 172.21.15.148:0/1740716285 shutdown_connections 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280b5470 0x7f60280b5900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 --2- 172.21.15.148:0/1740716285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f602815de00 0x7f60280b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 -- 172.21.15.148:0/1740716285 >> 172.21.15.148:0/1740716285 conn(0x7f60280d0470 msgr2=0x7f60280d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 -- 172.21.15.148:0/1740716285 shutdown_connections 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.376+0000 7f602df7e640 1 -- 172.21.15.148:0/1740716285 wait complete. 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.377+0000 7f602df7e640 1 Processor -- start 2024-03-04T12:12:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.377+0000 7f602df7e640 1 -- start start 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 0x7f60280cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280b9380 0x7f602815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280b97e0 con 0x7f602815de00 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280d24b0 con 0x7f60280b5470 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f602df7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280b72a0 con 0x7f60280b9380 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59144/0 (socket says 172.21.15.148:59144) 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 -- 172.21.15.148:0/2289019453 learned_addr learned my addr 172.21.15.148:0/2289019453 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6026ffd640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280b9380 0x7f602815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f60277fe640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 0x7f60280cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 -- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280b9380 msgr2=0x7f602815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280b9380 0x7f602815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 -- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 msgr2=0x7f60280cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 0x7f60280cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f6027fff640 1 -- 172.21.15.148:0/2289019453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602815ff30 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.378+0000 7f60277fe640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 0x7f60280cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f6027fff640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6020036040 tx=0x7f6020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020043020 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602003e450 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020009510 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028160210 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.379+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60281607d0 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.380+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028074f60 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.380+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6020046020 con 0x7f602815de00 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.381+0000 7f600ffff640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 0x7f601406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.381+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f60200ace90 con 0x7f602815de00 2024-03-04T12:12:51.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.381+0000 7f60277fe640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 0x7f601406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.381+0000 7f60277fe640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 0x7f601406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60280b81c0 tx=0x7f6018008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.385+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f602004d030 con 0x7f602815de00 2024-03-04T12:12:51.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.567+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60280b8250 con 0x7f602815de00 2024-03-04T12:12:51.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.567+0000 7f600ffff640 1 -- 172.21.15.148:0/2289019453 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f602007b9a0 con 0x7f602815de00 2024-03-04T12:12:51.569 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T12:12:51.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 msgr2=0x7f601406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 0x7f601406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60280b81c0 tx=0x7f6018008040 comp rx=0 tx=0).stop 2024-03-04T12:12:51.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 msgr2=0x7f60280caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6020036040 tx=0x7f6020004a40 comp rx=0 tx=0).stop 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 shutdown_connections 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f601406b4d0 0x7f601406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f602815de00 0x7f60280caac0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280b9380 0x7f602815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 --2- 172.21.15.148:0/2289019453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280b5470 0x7f60280cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 >> 172.21.15.148:0/2289019453 conn(0x7f60280d0470 msgr2=0x7f602806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 shutdown_connections 2024-03-04T12:12:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.569+0000 7f602df7e640 1 -- 172.21.15.148:0/2289019453 wait complete. 2024-03-04T12:12:51.582 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-04T12:12:51.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:51.583 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-04T12:12:51.616 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T12:12:51.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:12:51.617 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-04T12:12:51.674 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-04T12:12:51.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:51.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 Processor -- start 2024-03-04T12:12:51.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 -- start start 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7a6c0d1290 0x7f7a6c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c0d0400 con 0x7f7a6c0cfae0 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6c06ae50 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a720e8640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6c06b130 con 0x7f7a6c0d1290 2024-03-04T12:12:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.774+0000 7f7a62ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59148/0 (socket says 172.21.15.148:59148) 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 -- 172.21.15.148:0/2406726647 learned_addr learned my addr 172.21.15.148:0/2406726647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a6bfff640 1 --2- 172.21.15.148:0/2406726647 >> v2:172.21.15.191:3300/0 conn(0x7f7a6c0d1290 0x7f7a6c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 -- 172.21.15.148:0/2406726647 >> v2:172.21.15.191:3300/0 conn(0x7f7a6c0d1290 msgr2=0x7f7a6c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 --2- 172.21.15.148:0/2406726647 >> v2:172.21.15.191:3300/0 conn(0x7f7a6c0d1290 0x7f7a6c0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 -- 172.21.15.148:0/2406726647 >> v1:172.21.15.191:6789/0 conn(0x7f7a6c0cfae0 legacy=0x7f7a6c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.775+0000 7f7a62ffd640 1 -- 172.21.15.148:0/2406726647 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c06b410 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a62ffd640 1 --2- 172.21.15.148:0/2406726647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c06cf40 tx=0x7f7a5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d5097afb88af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a6affd640 1 -- 172.21.15.148:0/2406726647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54035020 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a6affd640 1 -- 172.21.15.148:0/2406726647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54031870 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a6affd640 1 -- 172.21.15.148:0/2406726647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54031c40 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 -- 172.21.15.148:0/2406726647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0cc8e0 msgr2=0x7f7a6c0cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 --2- 172.21.15.148:0/2406726647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c06cf40 tx=0x7f7a5402b910 comp rx=0 tx=0).stop 2024-03-04T12:12:51.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 -- 172.21.15.148:0/2406726647 shutdown_connections 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 --2- 172.21.15.148:0/2406726647 >> v2:172.21.15.191:3300/0 conn(0x7f7a6c0d1290 0x7f7a6c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 --2- 172.21.15.148:0/2406726647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c0cfec0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 -- 172.21.15.148:0/2406726647 >> 172.21.15.148:0/2406726647 conn(0x7f7a6c0d0770 msgr2=0x7f7a6c0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 -- 172.21.15.148:0/2406726647 shutdown_connections 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.776+0000 7f7a720e8640 1 -- 172.21.15.148:0/2406726647 wait complete. 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.777+0000 7f7a720e8640 1 Processor -- start 2024-03-04T12:12:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 -- start start 2024-03-04T12:12:51.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 0x7f7a6c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c06b9d0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0ce440 con 0x7f7a6c0cc8e0 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a720e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0cd980 con 0x7f7a6c0cfae0 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a6bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a6bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59156/0 (socket says 172.21.15.148:59156) 2024-03-04T12:12:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.778+0000 7f7a6bfff640 1 -- 172.21.15.148:0/3118407917 learned_addr learned my addr 172.21.15.148:0/3118407917 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6b7fe640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a62ffd640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 0x7f7a6c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 -- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 msgr2=0x7f7a6c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 0x7f7a6c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 -- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 msgr2=0x7f7a6c1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c1527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 -- 172.21.15.148:0/3118407917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0b5030 con 0x7f7a6c0d1290 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6b7fe640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a62ffd640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 0x7f7a6c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a6bfff640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a64007e60 tx=0x7f7a6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64014420 con 0x7f7a6c0d1290 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64019070 con 0x7f7a6c0d1290 2024-03-04T12:12:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c0b5310 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64014820 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.779+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c0b58d0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.780+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c06cd70 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.781+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7a640149c0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.781+0000 7f7a68ff9640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.781+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7a6408adc0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.781+0000 7f7a6b7fe640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.781+0000 7f7a6b7fe640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c003eb0 tx=0x7f7a5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.785+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a640598d0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.994+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a6c074f60 con 0x7f7a6c0d1290 2024-03-04T12:12:51.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.995+0000 7f7a68ff9640 1 -- 172.21.15.148:0/3118407917 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f7a640591d0 con 0x7f7a6c0d1290 2024-03-04T12:12:51.996 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:51.997 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:51.997 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:51.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 msgr2=0x7f7a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c003eb0 tx=0x7f7a5c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 msgr2=0x7f7a6c153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a64007e60 tx=0x7f7a6400e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 shutdown_connections 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6c0d1290 0x7f7a6c153710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a6c0cfae0 0x7f7a6c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 --2- 172.21.15.148:0/3118407917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a6c0cc8e0 0x7f7a6c1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 >> 172.21.15.148:0/3118407917 conn(0x7f7a6c0d0770 msgr2=0x7f7a6c15acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 shutdown_connections 2024-03-04T12:12:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:51.997+0000 7f7a720e8640 1 -- 172.21.15.148:0/3118407917 wait complete. 2024-03-04T12:12:52.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:52.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.109+0000 7f2f9ccb8640 1 Processor -- start 2024-03-04T12:12:52.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.109+0000 7f2f9ccb8640 1 -- start start 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9ccb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f9815dea0 0x7f2f980cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9ccb8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9806b3e0 0x7f2f980cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9ccb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9806fd20 con 0x7f2f980cc8d0 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9ccb8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9806fad0 con 0x7f2f9815dea0 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9ccb8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98070000 con 0x7f2f9806b3e0 2024-03-04T12:12:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f9815dea0 0x7f2f980cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f95d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2f980cc8d0 0x7f2f9806acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50262/0 (socket says 172.21.15.148:50262) 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f95d9b640 1 -- 172.21.15.148:0/927167158 learned_addr learned my addr 172.21.15.148:0/927167158 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f96d9d640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9806b3e0 0x7f2f980cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9659c640 1 -- 172.21.15.148:0/927167158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9806b3e0 msgr2=0x7f2f980cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9659c640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9806b3e0 0x7f2f980cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9659c640 1 -- 172.21.15.148:0/927167158 >> v1:172.21.15.148:6789/0 conn(0x7f2f980cc8d0 legacy=0x7f2f9806acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9659c640 1 -- 172.21.15.148:0/927167158 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980702e0 con 0x7f2f9815dea0 2024-03-04T12:12:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.110+0000 7f2f9559a640 1 -- 172.21.15.148:0/927167158 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250690530 0 0) 0x7f2f9806fd20 con 0x7f2f980cc8d0 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.111+0000 7f2f9659c640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980cad60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2f880097a0 tx=0x7f2f8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52954e1c4659a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.111+0000 7f2f9559a640 1 -- 172.21.15.148:0/927167158 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88035020 con 0x7f2f9815dea0 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.111+0000 7f2f9559a640 1 -- 172.21.15.148:0/927167158 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88031830 con 0x7f2f9815dea0 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.111+0000 7f2f9559a640 1 -- 172.21.15.148:0/927167158 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88031c00 con 0x7f2f9815dea0 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/927167158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 msgr2=0x7f2f980cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980cad60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2f880097a0 tx=0x7f2f8802b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/927167158 shutdown_connections 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9806b3e0 0x7f2f980cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/927167158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980cad60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/927167158 >> 172.21.15.148:0/927167158 conn(0x7f2f980d0710 msgr2=0x7f2f980d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/927167158 shutdown_connections 2024-03-04T12:12:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.112+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/927167158 wait complete. 2024-03-04T12:12:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.113+0000 7f2f9ccb8640 1 Processor -- start 2024-03-04T12:12:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.113+0000 7f2f9ccb8640 1 -- start start 2024-03-04T12:12:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 0x7f2f98150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 0x7f2f980c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d1380 con 0x7f2f9815dea0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d1200 con 0x7f2f9806b3e0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9ccb8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d1500 con 0x7f2f980cc8d0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59164/0 (socket says 172.21.15.148:59164) 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 -- 172.21.15.148:0/700883024 learned_addr learned my addr 172.21.15.148:0/700883024 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9659c640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 0x7f2f98150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f95d9b640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 0x7f2f980c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 -- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 msgr2=0x7f2f980c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 0x7f2f980c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 -- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 msgr2=0x7f2f98150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 0x7f2f98150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f96d9d640 1 -- 172.21.15.148:0/700883024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980c6260 con 0x7f2f9815dea0 2024-03-04T12:12:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f95d9b640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 0x7f2f980c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.114+0000 7f2f9659c640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 0x7f2f98150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f96d9d640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f90007d40 tx=0x7f2f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90014420 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90019070 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980c4c80 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90014820 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.115+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980c5240 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.116+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98074f60 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.116+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2f900149c0 con 0x7f2f9815dea0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.117+0000 7f2f86ffd640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 0x7f2f7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.117+0000 7f2f9659c640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 0x7f2f7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.117+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2f9008ae50 con 0x7f2f9815dea0 2024-03-04T12:12:52.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.117+0000 7f2f9659c640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 0x7f2f7806d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f88036000 tx=0x7f2f88008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.121+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f900598d0 con 0x7f2f9815dea0 2024-03-04T12:12:52.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.294+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f980cef00 con 0x7f2f9815dea0 2024-03-04T12:12:52.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.295+0000 7f2f86ffd640 1 -- 172.21.15.148:0/700883024 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f2f900591d0 con 0x7f2f9815dea0 2024-03-04T12:12:52.297 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:52.297 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 msgr2=0x7f2f7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 0x7f2f7806d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f88036000 tx=0x7f2f88008030 comp rx=0 tx=0).stop 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 msgr2=0x7f2f980c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f90007d40 tx=0x7f2f9000e4b0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 shutdown_connections 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2f7806b4d0 0x7f2f7806d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9815dea0 0x7f2f980c4200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f980cc8d0 0x7f2f980c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 --2- 172.21.15.148:0/700883024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f9806b3e0 0x7f2f98150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 >> 172.21.15.148:0/700883024 conn(0x7f2f980d0710 msgr2=0x7f2f980d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 shutdown_connections 2024-03-04T12:12:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.297+0000 7f2f9ccb8640 1 -- 172.21.15.148:0/700883024 wait complete. 2024-03-04T12:12:52.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:11:52.775816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:11:53.193920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T12:11:56.927475+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T12:12:52.310 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:12:52.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 Processor -- start 2024-03-04T12:12:52.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 -- start start 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f69700cc900 0x7f697015d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697006ae00 con 0x7f69700cdd70 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697006b0e0 con 0x7f69700ccfe0 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.414+0000 7f69771ec640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700ccde0 con 0x7f69700cc900 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f69700cdd70 0x7f69700cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50278/0 (socket says 172.21.15.148:50278) 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 -- 172.21.15.148:0/471062272 learned_addr learned my addr 172.21.15.148:0/471062272 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696ffff640 1 --2- 172.21.15.148:0/471062272 >> v2:172.21.15.191:3300/0 conn(0x7f69700cc900 0x7f697015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345188116 0 0) 0x7f697006ae00 con 0x7f69700cdd70 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c003520 con 0x7f69700cdd70 2024-03-04T12:12:52.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037131495 0 0) 0x7f695c003520 con 0x7f69700cdd70 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 >> v2:172.21.15.191:3300/0 conn(0x7f69700cc900 msgr2=0x7f697015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 --2- 172.21.15.148:0/471062272 >> v2:172.21.15.191:3300/0 conn(0x7f69700cc900 0x7f697015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 >> v1:172.21.15.191:6789/0 conn(0x7f69700ccfe0 legacy=0x7f697015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697006b3c0 con 0x7f69700cdd70 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f6960001670 con 0x7f69700cdd70 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 >> v1:172.21.15.148:6789/0 conn(0x7f69700cdd70 legacy=0x7f69700cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700ccde0 con 0x7f695c0039b0 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c160 con 0x7f695c0051c0 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c360 con 0x7f695c004800 2024-03-04T12:12:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6975789640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f696ffff640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 -- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f695c004800 msgr2=0x7f695c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 -- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f695c0051c0 msgr2=0x7f695c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.415+0000 7f6974f88640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f6974f88640 1 -- 172.21.15.148:0/471062272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c003520 con 0x7f695c0039b0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f6975789640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f6975789640 1 -- 172.21.15.148:0/471062272 reap_dead start 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f6974f88640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f696000c210 tx=0x7f695c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ed0b5e5959091f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960007d30 con 0x7f695c0039b0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6960030070 con 0x7f695c0039b0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.416+0000 7f696f7fe640 1 -- 172.21.15.148:0/471062272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696002cda0 con 0x7f695c0039b0 2024-03-04T12:12:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 -- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 msgr2=0x7f695c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f696000c210 tx=0x7f695c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 -- 172.21.15.148:0/471062272 shutdown_connections 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 --2- 172.21.15.148:0/471062272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 -- 172.21.15.148:0/471062272 >> 172.21.15.148:0/471062272 conn(0x7f69700b22e0 msgr2=0x7f69700b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 -- 172.21.15.148:0/471062272 shutdown_connections 2024-03-04T12:12:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.417+0000 7f69771ec640 1 -- 172.21.15.148:0/471062272 wait complete. 2024-03-04T12:12:52.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.418+0000 7f69771ec640 1 Processor -- start 2024-03-04T12:12:52.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.418+0000 7f69771ec640 1 -- start start 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 0x7f69701e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69701e92f0 0x7f69700c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b3010 con 0x7f695c0039b0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b2e90 con 0x7f69700c54a0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f69771ec640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700af950 con 0x7f69701e92f0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59182/0 (socket says 172.21.15.148:59182) 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 -- 172.21.15.148:0/2656448794 learned_addr learned my addr 172.21.15.148:0/2656448794 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6975789640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69701e92f0 0x7f69700c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f696ffff640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 0x7f69701e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 -- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69701e92f0 msgr2=0x7f69700c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69701e92f0 0x7f69700c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 -- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 msgr2=0x7f69701e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 0x7f69701e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f6974f88640 1 -- 172.21.15.148:0/2656448794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69701f7a40 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.419+0000 7f696ffff640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 0x7f69701e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f6974f88640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f696000c330 tx=0x7f696002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960004bf0 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6960004d90 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960007270 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69701f7d20 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.420+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69701f82e0 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.421+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69701f8720 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.421+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6960048020 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.422+0000 7f696d7fa640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 0x7f695006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.422+0000 7f696ffff640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 0x7f695006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.422+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f69600aba10 con 0x7f695c0039b0 2024-03-04T12:12:52.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.422+0000 7f696ffff640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 0x7f695006d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69700c4ac0 tx=0x7f6958008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.426+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f696007a520 con 0x7f695c0039b0 2024-03-04T12:12:52.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.596+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69701f8720 con 0x7f695c0039b0 2024-03-04T12:12:52.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.596+0000 7f696d7fa640 1 -- 172.21.15.148:0/2656448794 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6960079e20 con 0x7f695c0039b0 2024-03-04T12:12:52.600 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 msgr2=0x7f695006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 0x7f695006d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69700c4ac0 tx=0x7f6958008040 comp rx=0 tx=0).stop 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 msgr2=0x7f69700c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f696000c330 tx=0x7f696002e070 comp rx=0 tx=0).stop 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 shutdown_connections 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f695006b500 0x7f695006d9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69701e92f0 0x7f69700c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69700c54a0 0x7f69701e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 --2- 172.21.15.148:0/2656448794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f695c0039b0 0x7f69700c4f60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 >> 172.21.15.148:0/2656448794 conn(0x7f69700b22e0 msgr2=0x7f69700cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 shutdown_connections 2024-03-04T12:12:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.598+0000 7f69771ec640 1 -- 172.21.15.148:0/2656448794 wait complete. 2024-03-04T12:12:52.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T12:12:52.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.713+0000 7ff8ae90a640 1 Processor -- start 2024-03-04T12:12:52.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.713+0000 7ff8ae90a640 1 -- start start 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8ae90a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a80b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8ae90a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8a80ceec0 0x7ff8a80b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8ae90a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a815ca50 con 0x7ff8a815c2e0 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8ae90a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a815d2f0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8ae90a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a815d6d0 con 0x7ff8a80ccbd0 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8acea7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8a80ceec0 0x7ff8a80b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8acea7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8a80ceec0 0x7ff8a80b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40182/0 (socket says 172.21.15.148:40182) 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.714+0000 7ff8acea7640 1 -- 172.21.15.148:0/2271698623 learned_addr learned my addr 172.21.15.148:0/2271698623 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8acea7640 1 -- 172.21.15.148:0/2271698623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 msgr2=0x7ff8a80b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8a7fff640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a80b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8acea7640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a80b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8acea7640 1 -- 172.21.15.148:0/2271698623 >> v1:172.21.15.191:6789/0 conn(0x7ff8a815c2e0 legacy=0x7ff8a80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8acea7640 1 -- 172.21.15.148:0/2271698623 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a815cd30 con 0x7ff8a80ceec0 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.715+0000 7ff8acea7640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80b0560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009e50 tx=0x7ff8a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985a6a294e785451 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8a6ffd640 1 -- 172.21.15.148:0/2271698623 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a002ca70 con 0x7ff8a80ceec0 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8a6ffd640 1 -- 172.21.15.148:0/2271698623 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a002cc10 con 0x7ff8a80ceec0 2024-03-04T12:12:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8a6ffd640 1 -- 172.21.15.148:0/2271698623 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00343f0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 -- 172.21.15.148:0/2271698623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 msgr2=0x7ff8a80b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80b0560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009e50 tx=0x7ff8a002bed0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 -- 172.21.15.148:0/2271698623 shutdown_connections 2024-03-04T12:12:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80b0560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/2271698623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a80b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 -- 172.21.15.148:0/2271698623 >> 172.21.15.148:0/2271698623 conn(0x7ff8a8066dc0 msgr2=0x7ff8a80671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 -- 172.21.15.148:0/2271698623 shutdown_connections 2024-03-04T12:12:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.716+0000 7ff8ae90a640 1 -- 172.21.15.148:0/2271698623 wait complete. 2024-03-04T12:12:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.717+0000 7ff8ae90a640 1 Processor -- start 2024-03-04T12:12:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.717+0000 7ff8ae90a640 1 -- start start 2024-03-04T12:12:52.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a8157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a815c2e0 0x7ff8a806bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a80ccfb0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a815c760 con 0x7ff8a815c2e0 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8ae90a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a80b0aa0 con 0x7ff8a80ccbd0 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8a77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8a77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40200/0 (socket says 172.21.15.148:40200) 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.718+0000 7ff8a77fe640 1 -- 172.21.15.148:0/1396443710 learned_addr learned my addr 172.21.15.148:0/1396443710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 -- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 msgr2=0x7ff8a8157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a8157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 -- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a815c2e0 msgr2=0x7ff8a806bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a815c2e0 0x7ff8a806bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 -- 172.21.15.148:0/1396443710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a806c9c0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a77fe640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b3f0 tx=0x7ff89c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c015020 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c011830 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a806cca0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c011c20 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.719+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a80c57f0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.720+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a8074f60 con 0x7ff8a80ceec0 2024-03-04T12:12:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.720+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff89c018020 con 0x7ff8a80ceec0 2024-03-04T12:12:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.721+0000 7ff8a4ff9640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 0x7ff89406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.721+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff89c08a290 con 0x7ff8a80ceec0 2024-03-04T12:12:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.721+0000 7ff8a7fff640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 0x7ff89406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.721+0000 7ff8a7fff640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 0x7ff89406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff898006fd0 tx=0x7ff898008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:52.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.725+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff89c01f030 con 0x7ff8a80ceec0 2024-03-04T12:12:52.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.895+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8a80c59c0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.895+0000 7ff8a4ff9640 1 -- 172.21.15.148:0/1396443710 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff89c0172c0 con 0x7ff8a80ceec0 2024-03-04T12:12:52.897 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.897+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 msgr2=0x7ff89406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 0x7ff89406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff898006fd0 tx=0x7ff898008040 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 msgr2=0x7ff8a80d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b3f0 tx=0x7ff89c00b8b0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 shutdown_connections 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff89406b4d0 0x7ff89406d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a815c2e0 0x7ff8a806bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a80ceec0 0x7ff8a80d1a60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 --2- 172.21.15.148:0/1396443710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a80ccbd0 0x7ff8a8157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 >> 172.21.15.148:0/1396443710 conn(0x7ff8a8066dc0 msgr2=0x7ff8a80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:52.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 shutdown_connections 2024-03-04T12:12:52.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:52.898+0000 7ff8ae90a640 1 -- 172.21.15.148:0/1396443710 wait complete. 2024-03-04T12:12:52.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T12:12:53.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.011+0000 7f3cfe6d1640 1 Processor -- start 2024-03-04T12:12:53.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.011+0000 7f3cfe6d1640 1 -- start start 2024-03-04T12:12:53.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfe6d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3cf80b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfe6d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf815ca50 con 0x7f3cf80ceec0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfe6d1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf815cd30 con 0x7f3cf815c2e0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfe6d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf815d2f0 con 0x7f3cf80ccbd0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfcc6e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3cf80ceec0 0x7f3cf80b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59376/0 (socket says 172.21.15.148:59376) 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.012+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2514224703 learned_addr learned my addr 172.21.15.148:0/2514224703 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf7fff640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3cf80b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223581772 0 0) 0x7f3cf815ca50 con 0x7f3cf80ceec0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce0003520 con 0x7f3cf80ceec0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619298668 0 0) 0x7f3ce0003520 con 0x7f3cf80ceec0 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 msgr2=0x7f3cf80b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3cf80b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 >> v1:172.21.15.191:6789/0 conn(0x7f3cf815c2e0 legacy=0x7f3cf80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf815d010 con 0x7f3cf80ceec0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f3cf00019f0 con 0x7f3cf80ceec0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 >> v1:172.21.15.148:6789/0 conn(0x7f3cf80ceec0 legacy=0x7f3cf80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce00039e0 0x7f3ce0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3ce0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce0004d40 0x7f3ce00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000b9c0 con 0x7f3ce0004d40 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000bb40 con 0x7f3ce00039e0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000bd40 con 0x7f3cf80ccbd0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce0004d40 0x7f3ce00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf7fff640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce00039e0 0x7f3ce0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cf77fe640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3ce0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 msgr2=0x7f3ce0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cf80ccbd0 0x7f3ce0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce00039e0 msgr2=0x7f3ce0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce00039e0 0x7f3ce0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.013+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2514224703 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0003520 con 0x7f3ce0004d40 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.014+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce0004d40 0x7f3ce00051c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000a810 tx=0x7f3ce0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9b6decce46c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.014+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000a430 con 0x7f3ce0004d40 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.014+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf0034630 con 0x7f3ce0004d40 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.014+0000 7f3cf6ffd640 1 -- 172.21.15.148:0/2514224703 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0034a20 con 0x7f3ce0004d40 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2514224703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce0004d40 msgr2=0x7f3ce00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2514224703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce0004d40 0x7f3ce00051c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000a810 tx=0x7f3ce0005830 comp rx=0 tx=0).stop 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2514224703 reap_dead start 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2514224703 shutdown_connections 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2514224703 >> 172.21.15.148:0/2514224703 conn(0x7f3cf8066dc0 msgr2=0x7f3cf80671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2514224703 shutdown_connections 2024-03-04T12:12:53.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.015+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2514224703 wait complete. 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.016+0000 7f3cfe6d1640 1 Processor -- start 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.016+0000 7f3cfe6d1640 1 -- start start 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0004d40 0x7f3cf8158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cf80c5d80 0x7f3cf806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80ccfb0 con 0x7f3cf806d0c0 2024-03-04T12:12:53.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf815d6d0 con 0x7f3cf80c5d80 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfe6d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf815eb70 con 0x7f3ce0004d40 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40206/0 (socket says 172.21.15.148:40206) 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2726909272 learned_addr learned my addr 172.21.15.148:0/2726909272 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0004d40 msgr2=0x7f3cf8158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0004d40 0x7f3cf8158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cf80c5d80 msgr2=0x7f3cf806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cf80c5d80 0x7f3cf806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 -- 172.21.15.148:0/2726909272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80c7aa0 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.017+0000 7f3cfcc6e640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0001c70 tx=0x7f3cf0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.018+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000a430 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.018+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf0044070 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.018+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf80c7d40 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.018+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0034c30 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.018+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf80c8b60 con 0x7f3cf806d0c0 2024-03-04T12:12:53.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.019+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf806c980 con 0x7f3cf806d0c0 2024-03-04T12:12:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.019+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3cf0049460 con 0x7f3cf806d0c0 2024-03-04T12:12:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.019+0000 7f3cf4ff9640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 0x7f3ce406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.020+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3cf00acb50 con 0x7f3cf806d0c0 2024-03-04T12:12:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.020+0000 7f3cf7fff640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 0x7f3ce406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.020+0000 7f3cf7fff640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 0x7f3ce406d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009ab0 tx=0x7f3ce80095d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.024+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cf806c980 con 0x7f3cf806d0c0 2024-03-04T12:12:53.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.194+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cf80cc5f0 con 0x7f3cf806d0c0 2024-03-04T12:12:53.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.195+0000 7f3cf4ff9640 1 -- 172.21.15.148:0/2726909272 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3cf80cc5f0 con 0x7f3cf806d0c0 2024-03-04T12:12:53.196 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 msgr2=0x7f3ce406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 0x7f3ce406d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009ab0 tx=0x7f3ce80095d0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 msgr2=0x7f3cf81f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0001c70 tx=0x7f3cf0001d40 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 shutdown_connections 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ce406b500 0x7f3ce406d9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf806d0c0 0x7f3cf81f4070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cf80c5d80 0x7f3cf806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 --2- 172.21.15.148:0/2726909272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0004d40 0x7f3cf8158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 >> 172.21.15.148:0/2726909272 conn(0x7f3cf8066dc0 msgr2=0x7f3cf806c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 shutdown_connections 2024-03-04T12:12:53.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.197+0000 7f3cfe6d1640 1 -- 172.21.15.148:0/2726909272 wait complete. 2024-03-04T12:12:53.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T12:12:53.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.310+0000 7ff075d09640 1 Processor -- start 2024-03-04T12:12:53.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.310+0000 7ff075d09640 1 -- start start 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff075d09640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff0700ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff075d09640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff07015e810 0x7ff07006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff075d09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07015bae0 con 0x7ff07006bce0 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff075d09640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07006c680 con 0x7ff07015e810 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff075d09640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700adb90 con 0x7ff07015b700 2024-03-04T12:12:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff07006bce0 0x7ff07015e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59390/0 (socket says 172.21.15.148:59390) 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06effd640 1 -- 172.21.15.148:0/1672756151 learned_addr learned my addr 172.21.15.148:0/1672756151 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff0700ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06ffff640 1 --2- 172.21.15.148:0/1672756151 >> v2:172.21.15.191:3300/0 conn(0x7ff07015e810 0x7ff07006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062793535 0 0) 0x7ff07015bae0 con 0x7ff07006bce0 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.311+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff058003520 con 0x7ff07006bce0 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4238613318 0 0) 0x7ff058003520 con 0x7ff07006bce0 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 msgr2=0x7ff0700ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff0700ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 >> v2:172.21.15.191:3300/0 conn(0x7ff07015e810 msgr2=0x7ff07006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 --2- 172.21.15.148:0/1672756151 >> v2:172.21.15.191:3300/0 conn(0x7ff07015e810 0x7ff07006c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07015ee60 con 0x7ff07006bce0 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7ff064001670 con 0x7ff07006bce0 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 >> v1:172.21.15.148:6789/0 conn(0x7ff07006bce0 legacy=0x7ff07015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580039b0 0x7ff058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff058004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058004d40 0x7ff0580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700adb90 con 0x7ff0580039b0 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05800b9c0 con 0x7ff058004d40 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05800bb40 con 0x7ff07015b700 2024-03-04T12:12:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580039b0 0x7ff058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06effd640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff058004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06ffff640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058004d40 0x7ff0580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 -- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 msgr2=0x7ff058004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff07015b700 0x7ff058004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 -- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058004d40 msgr2=0x7ff0580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058004d40 0x7ff0580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06f7fe640 1 -- 172.21.15.148:0/1672756151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058003520 con 0x7ff0580039b0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.312+0000 7ff06ffff640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058004d40 0x7ff0580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580039b0 0x7ff058003dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff060009ac0 tx=0x7ff058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82383c286b0c2cad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060039070 con 0x7ff0580039b0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06002c6b0 con 0x7ff0580039b0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff06e7fc640 1 -- 172.21.15.148:0/1672756151 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060009690 con 0x7ff0580039b0 2024-03-04T12:12:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff075d09640 1 -- 172.21.15.148:0/1672756151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580039b0 msgr2=0x7ff058003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff075d09640 1 --2- 172.21.15.148:0/1672756151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580039b0 0x7ff058003dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff060009ac0 tx=0x7ff058005830 comp rx=0 tx=0).stop 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff06ffff640 1 -- 172.21.15.148:0/1672756151 reap_dead start 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.313+0000 7ff075d09640 1 -- 172.21.15.148:0/1672756151 shutdown_connections 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.314+0000 7ff075d09640 1 -- 172.21.15.148:0/1672756151 >> 172.21.15.148:0/1672756151 conn(0x7ff070071bc0 msgr2=0x7ff070071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.314+0000 7ff075d09640 1 -- 172.21.15.148:0/1672756151 shutdown_connections 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.314+0000 7ff075d09640 1 -- 172.21.15.148:0/1672756151 wait complete. 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.315+0000 7ff075d09640 1 Processor -- start 2024-03-04T12:12:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.315+0000 7ff075d09640 1 -- start start 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0580039b0 0x7ff0701f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff07014e420 0x7ff07014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700ae020 con 0x7ff0701480a0 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700ad820 con 0x7ff07014e420 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700ae1a0 con 0x7ff0580039b0 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40234/0 (socket says 172.21.15.148:40234) 2024-03-04T12:12:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 -- 172.21.15.148:0/350603958 learned_addr learned my addr 172.21.15.148:0/350603958 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06effd640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff07014e420 0x7ff07014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0580039b0 0x7ff0701f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 -- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0580039b0 msgr2=0x7ff0701f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0580039b0 0x7ff0701f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 -- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff07014e420 msgr2=0x7ff07014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff07014e420 0x7ff07014b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 -- 172.21.15.148:0/350603958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07014b8b0 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff06ffff640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff0700671c0 tx=0x7ff068010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06801a030 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068010d80 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0700cc970 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068003bd0 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.316+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0701f6d70 con 0x7ff0701480a0 2024-03-04T12:12:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.317+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff070074f60 con 0x7ff0701480a0 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.318+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff068005d10 con 0x7ff0701480a0 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.318+0000 7ff057fff640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 0x7ff05c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.318+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff06808abe0 con 0x7ff0701480a0 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.318+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 0x7ff05c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.318+0000 7ff06f7fe640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 0x7ff05c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff060036000 tx=0x7ff060030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.322+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff068025020 con 0x7ff0701480a0 2024-03-04T12:12:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.493+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff070073b20 con 0x7ff0701480a0 2024-03-04T12:12:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.494+0000 7ff057fff640 1 -- 172.21.15.148:0/350603958 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff07014b8b0 con 0x7ff0701480a0 2024-03-04T12:12:53.496 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T12:12:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.495+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 msgr2=0x7ff05c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.495+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 0x7ff05c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff060036000 tx=0x7ff060030000 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.495+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 msgr2=0x7ff070158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.495+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff0700671c0 tx=0x7ff068010590 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 shutdown_connections 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff05c06b4d0 0x7ff05c06d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0701480a0 0x7ff070158b90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff07014e420 0x7ff07014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 --2- 172.21.15.148:0/350603958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0580039b0 0x7ff0701f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 >> 172.21.15.148:0/350603958 conn(0x7ff070071bc0 msgr2=0x7ff0700b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 shutdown_connections 2024-03-04T12:12:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.496+0000 7ff075d09640 1 -- 172.21.15.148:0/350603958 wait complete. 2024-03-04T12:12:53.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:53.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.607+0000 7f48482e8640 1 Processor -- start 2024-03-04T12:12:53.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.607+0000 7f48482e8640 1 -- start start 2024-03-04T12:12:53.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f48482e8640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f484006d3b0 0x7f48400ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f48482e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48400cc500 0x7f48400cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f48482e8640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400d0140 con 0x7f48400cfd60 2024-03-04T12:12:53.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f48482e8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400cf440 con 0x7f48400cc500 2024-03-04T12:12:53.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f48482e8640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484015cbb0 con 0x7f484006d3b0 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f4846885640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48400cc500 0x7f48400cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f4846885640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48400cc500 0x7f48400cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40236/0 (socket says 172.21.15.148:40236) 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f4846885640 1 -- 172.21.15.148:0/1625476875 learned_addr learned my addr 172.21.15.148:0/1625476875 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.608+0000 7f4845883640 1 --2- 172.21.15.148:0/1625476875 >> v2:172.21.15.191:3300/0 conn(0x7f484006d3b0 0x7f48400ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.609+0000 7f4846885640 1 -- 172.21.15.148:0/1625476875 >> v2:172.21.15.191:3300/0 conn(0x7f484006d3b0 msgr2=0x7f48400ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.609+0000 7f4846885640 1 --2- 172.21.15.148:0/1625476875 >> v2:172.21.15.191:3300/0 conn(0x7f484006d3b0 0x7f48400ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.609+0000 7f4846885640 1 -- 172.21.15.148:0/1625476875 >> v1:172.21.15.191:6789/0 conn(0x7f48400cfd60 legacy=0x7f48400cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.609+0000 7f4846885640 1 -- 172.21.15.148:0/1625476875 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484015c280 con 0x7f48400cc500 2024-03-04T12:12:53.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.609+0000 7f4846885640 1 --2- 172.21.15.148:0/1625476875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400cef00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f484015cd30 tx=0x7f483802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d4d21e2b0d7423 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f4845082640 1 -- 172.21.15.148:0/1625476875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838034070 con 0x7f48400cc500 2024-03-04T12:12:53.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f4845082640 1 -- 172.21.15.148:0/1625476875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838030860 con 0x7f48400cc500 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f4845082640 1 -- 172.21.15.148:0/1625476875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838030c30 con 0x7f48400cc500 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 -- 172.21.15.148:0/1625476875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 msgr2=0x7f48400cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 --2- 172.21.15.148:0/1625476875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400cef00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f484015cd30 tx=0x7f483802b9a0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 -- 172.21.15.148:0/1625476875 shutdown_connections 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 --2- 172.21.15.148:0/1625476875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400cef00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 --2- 172.21.15.148:0/1625476875 >> v2:172.21.15.191:3300/0 conn(0x7f484006d3b0 0x7f48400ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 -- 172.21.15.148:0/1625476875 >> 172.21.15.148:0/1625476875 conn(0x7f484006acd0 msgr2=0x7f484006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 -- 172.21.15.148:0/1625476875 shutdown_connections 2024-03-04T12:12:53.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.610+0000 7f48482e8640 1 -- 172.21.15.148:0/1625476875 wait complete. 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.611+0000 7f48482e8640 1 Processor -- start 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.611+0000 7f48482e8640 1 -- start start 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f484006d3b0 0x7f4840146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48400cfd60 0x7f48400a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484015bf30 con 0x7f48400cc500 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484015c840 con 0x7f484006d3b0 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f48482e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484015f520 con 0x7f48400cfd60 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4845883640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4845883640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40238/0 (socket says 172.21.15.148:40238) 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4845883640 1 -- 172.21.15.148:0/3362344139 learned_addr learned my addr 172.21.15.148:0/3362344139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4846084640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f484006d3b0 0x7f4840146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4846885640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48400cfd60 0x7f48400a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4845883640 1 -- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48400cfd60 msgr2=0x7f48400a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.612+0000 7f4845883640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48400cfd60 0x7f48400a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4845883640 1 -- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f484006d3b0 msgr2=0x7f4840146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4845883640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f484006d3b0 0x7f4840146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4845883640 1 -- 172.21.15.148:0/3362344139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840146840 con 0x7f48400cc500 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4845883640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f483000e980 tx=0x7f483000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830017070 con 0x7f48400cc500 2024-03-04T12:12:53.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000ca00 con 0x7f48400cc500 2024-03-04T12:12:53.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000cdd0 con 0x7f48400cc500 2024-03-04T12:12:53.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401573e0 con 0x7f48400cc500 2024-03-04T12:12:53.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.613+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401579a0 con 0x7f48400cc500 2024-03-04T12:12:53.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.614+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840074f60 con 0x7f48400cc500 2024-03-04T12:12:53.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.614+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f483001a070 con 0x7f48400cc500 2024-03-04T12:12:53.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.615+0000 7f4836ffd640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 0x7f482406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.615+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f483008a570 con 0x7f48400cc500 2024-03-04T12:12:53.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.615+0000 7f4846084640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 0x7f482406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.615+0000 7f4846084640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 0x7f482406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4840154f70 tx=0x7f483c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.619+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4830059080 con 0x7f48400cc500 2024-03-04T12:12:53.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.825+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4840157000 con 0x7f48400cc500 2024-03-04T12:12:53.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.825+0000 7f4836ffd640 1 -- 172.21.15.148:0/3362344139 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f4830058980 con 0x7f48400cc500 2024-03-04T12:12:53.828 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:53.828 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 msgr2=0x7f482406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 0x7f482406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4840154f70 tx=0x7f483c009040 comp rx=0 tx=0).stop 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 msgr2=0x7f48400a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f483000e980 tx=0x7f483000ee40 comp rx=0 tx=0).stop 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 shutdown_connections 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f482406b4d0 0x7f482406d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48400cfd60 0x7f48400a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48400cc500 0x7f48400a8460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 --2- 172.21.15.148:0/3362344139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f484006d3b0 0x7f4840146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 >> 172.21.15.148:0/3362344139 conn(0x7f484006acd0 msgr2=0x7f484015b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 shutdown_connections 2024-03-04T12:12:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.827+0000 7f48482e8640 1 -- 172.21.15.148:0/3362344139 wait complete. 2024-03-04T12:12:53.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:53.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.939+0000 7f42cb6ea640 1 Processor -- start 2024-03-04T12:12:53.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.939+0000 7f42cb6ea640 1 -- start start 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42cb6ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f42c415bbb0 0x7f42c40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42cb6ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40cfb40 con 0x7f42c40d0e40 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42cb6ea640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40cfe20 con 0x7f42c40cc690 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42cb6ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40d2880 con 0x7f42c415bbb0 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c9486640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f42c415bbb0 0x7f42c40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c8c85640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f42c40d0e40 0x7f42c40cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59398/0 (socket says 172.21.15.148:59398) 2024-03-04T12:12:53.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c8c85640 1 -- 172.21.15.148:0/1729062653 learned_addr learned my addr 172.21.15.148:0/1729062653 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:53.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c9486640 1 -- 172.21.15.148:0/1729062653 >> v1:172.21.15.191:6789/0 conn(0x7f42c40cc690 legacy=0x7f42c40d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:53.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42bbfff640 1 -- 172.21.15.148:0/1729062653 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096119423 0 0) 0x7f42c40cfb40 con 0x7f42c40d0e40 2024-03-04T12:12:53.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c9486640 1 -- 172.21.15.148:0/1729062653 >> v1:172.21.15.148:6789/0 conn(0x7f42c40d0e40 legacy=0x7f42c40cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.940+0000 7f42c9486640 1 -- 172.21.15.148:0/1729062653 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40d0100 con 0x7f42c415bbb0 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.941+0000 7f42c9486640 1 --2- 172.21.15.148:0/1729062653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c40d0900 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f42c00097a0 tx=0x7f42c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c60839b973f3ddb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.941+0000 7f42bbfff640 1 -- 172.21.15.148:0/1729062653 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0034070 con 0x7f42c415bbb0 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.941+0000 7f42bbfff640 1 -- 172.21.15.148:0/1729062653 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0030610 con 0x7f42c415bbb0 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.941+0000 7f42bbfff640 1 -- 172.21.15.148:0/1729062653 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00309e0 con 0x7f42c415bbb0 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 -- 172.21.15.148:0/1729062653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 msgr2=0x7f42c40d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/1729062653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c40d0900 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f42c00097a0 tx=0x7f42c002b960 comp rx=0 tx=0).stop 2024-03-04T12:12:53.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 -- 172.21.15.148:0/1729062653 shutdown_connections 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/1729062653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c40d0900 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 -- 172.21.15.148:0/1729062653 >> 172.21.15.148:0/1729062653 conn(0x7f42c40b3480 msgr2=0x7f42c40b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 -- 172.21.15.148:0/1729062653 shutdown_connections 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.942+0000 7f42cb6ea640 1 -- 172.21.15.148:0/1729062653 wait complete. 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.943+0000 7f42cb6ea640 1 Processor -- start 2024-03-04T12:12:53.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.943+0000 7f42cb6ea640 1 -- start start 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c40cc690 0x7f42c40d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42c40d0e40 0x7f42c40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c415bf90 con 0x7f42c415bbb0 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40d23b0 con 0x7f42c40cc690 2024-03-04T12:12:53.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42cb6ea640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40caad0 con 0x7f42c40d0e40 2024-03-04T12:12:53.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40252/0 (socket says 172.21.15.148:40252) 2024-03-04T12:12:53.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 -- 172.21.15.148:0/3134060652 learned_addr learned my addr 172.21.15.148:0/3134060652 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:53.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c8c85640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42c40d0e40 0x7f42c40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 -- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42c40d0e40 msgr2=0x7f42c40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9486640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c40cc690 0x7f42c40d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42c40d0e40 0x7f42c40ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 -- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c40cc690 msgr2=0x7f42c40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c40cc690 0x7f42c40d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:53.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 -- 172.21.15.148:0/3134060652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40cad00 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.944+0000 7f42c9c87640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c730 tx=0x7f42bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.945+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc003b70 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.945+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc003d10 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.945+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c40cafe0 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.945+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00aa90 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.945+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c40a8760 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.946+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c40cb500 con 0x7f42c415bbb0 2024-03-04T12:12:53.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.946+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f42bc00ac30 con 0x7f42c415bbb0 2024-03-04T12:12:53.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.946+0000 7f42b9ffb640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 0x7f42a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:53.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.947+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f42bc08a4e0 con 0x7f42c415bbb0 2024-03-04T12:12:53.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.947+0000 7f42c9486640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 0x7f42a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:53.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.947+0000 7f42c9486640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 0x7f42a806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:53.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:53.951+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42c40cb500 con 0x7f42c415bbb0 2024-03-04T12:12:54.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.125+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c40cc4b0 con 0x7f42c415bbb0 2024-03-04T12:12:54.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.126+0000 7f42b9ffb640 1 -- 172.21.15.148:0/3134060652 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f42c40cc4b0 con 0x7f42c415bbb0 2024-03-04T12:12:54.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:54.129 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 msgr2=0x7f42a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 0x7f42a806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0008030 comp rx=0 tx=0).stop 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 msgr2=0x7f42c414cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c730 tx=0x7f42bc00cbf0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 shutdown_connections 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f42a806b4d0 0x7f42a806d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c415bbb0 0x7f42c414cc50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42c40d0e40 0x7f42c40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 --2- 172.21.15.148:0/3134060652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c40cc690 0x7f42c40d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 >> 172.21.15.148:0/3134060652 conn(0x7f42c40b3480 msgr2=0x7f42c40b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 shutdown_connections 2024-03-04T12:12:54.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.129+0000 7f42cb6ea640 1 -- 172.21.15.148:0/3134060652 wait complete. 2024-03-04T12:12:54.143 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-04T12:12:54.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.241+0000 7f3df939d640 1 Processor -- start 2024-03-04T12:12:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.241+0000 7f3df939d640 1 -- start start 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df939d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3df415d340 0x7f3df4099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df939d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df415c620 con 0x7f3df40b2a90 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df939d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df415c900 con 0x7f3df4099f40 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df939d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df415cbe0 con 0x7f3df415d340 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3df40b2a90 0x7f3df409a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59400/0 (socket says 172.21.15.148:59400) 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df3fff640 1 -- 172.21.15.148:0/1489601599 learned_addr learned my addr 172.21.15.148:0/1489601599 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df37fe640 1 --2- 172.21.15.148:0/1489601599 >> v2:172.21.15.191:3300/0 conn(0x7f3df415d340 0x7f3df4099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800774804 0 0) 0x7f3df415c620 con 0x7f3df40b2a90 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ddc003520 con 0x7f3df40b2a90 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 988850240 0 0) 0x7f3ddc003520 con 0x7f3df40b2a90 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 >> v2:172.21.15.191:3300/0 conn(0x7f3df415d340 msgr2=0x7f3df4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/1489601599 >> v2:172.21.15.191:3300/0 conn(0x7f3df415d340 0x7f3df4099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 >> v1:172.21.15.191:6789/0 conn(0x7f3df4099f40 legacy=0x7f3df40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.242+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df415cec0 con 0x7f3df40b2a90 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f3de4001670 con 0x7f3df40b2a90 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 >> v1:172.21.15.148:6789/0 conn(0x7f3df40b2a90 legacy=0x7f3df409a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc0039b0 0x7f3ddc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc0043b0 0x7f3ddc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3ddc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df415cbe0 con 0x7f3ddc005080 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc00bfe0 con 0x7f3ddc0043b0 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc00c1e0 con 0x7f3ddc0039b0 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3ddc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df37fe640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc0043b0 0x7f3ddc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df3fff640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc0039b0 0x7f3ddc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 -- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc0039b0 msgr2=0x7f3ddc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc0039b0 0x7f3ddc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 -- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc0043b0 msgr2=0x7f3ddc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc0043b0 0x7f3ddc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df8b9c640 1 -- 172.21.15.148:0/1489601599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc003520 con 0x7f3ddc005080 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.243+0000 7f3df37fe640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc0043b0 0x7f3ddc004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df8b9c640 1 -- 172.21.15.148:0/1489601599 reap_dead start 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df8b9c640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3ddc00b9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3dec001f60 tx=0x7f3ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92535ed3f5a246eb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8009690 con 0x7f3ddc005080 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec003b90 con 0x7f3ddc005080 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df2ffd640 1 -- 172.21.15.148:0/1489601599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec001b30 con 0x7f3ddc005080 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df939d640 1 -- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 msgr2=0x7f3ddc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.244+0000 7f3df939d640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3ddc00b9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3dec001f60 tx=0x7f3ddc005830 comp rx=0 tx=0).stop 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.245+0000 7f3df939d640 1 -- 172.21.15.148:0/1489601599 shutdown_connections 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.245+0000 7f3df939d640 1 --2- 172.21.15.148:0/1489601599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3ddc00b9c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.245+0000 7f3df939d640 1 -- 172.21.15.148:0/1489601599 >> 172.21.15.148:0/1489601599 conn(0x7f3df40b30e0 msgr2=0x7f3df40b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.245+0000 7f3df939d640 1 -- 172.21.15.148:0/1489601599 shutdown_connections 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.245+0000 7f3df939d640 1 -- 172.21.15.148:0/1489601599 wait complete. 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.246+0000 7f3df939d640 1 Processor -- start 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.246+0000 7f3df939d640 1 -- start start 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 0x7f3df41f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3df41f3bb0 0x7f3df40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4099a00 con 0x7f3ddc005080 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b2e70 con 0x7f3df41f4680 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4099b80 con 0x7f3df41f3bb0 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40270/0 (socket says 172.21.15.148:40270) 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 -- 172.21.15.148:0/3004386252 learned_addr learned my addr 172.21.15.148:0/3004386252 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 0x7f3df41f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df8b9c640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3df41f3bb0 0x7f3df40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 -- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3df41f3bb0 msgr2=0x7f3df40cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3df41f3bb0 0x7f3df40cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 -- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 msgr2=0x7f3df41f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 0x7f3df41f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 -- 172.21.15.148:0/3004386252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40c9c00 con 0x7f3ddc005080 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 0x7f3df41f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3de40011e0 tx=0x7f3de40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400f9c0 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4018070 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41f40c0 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400fd80 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.247+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df40c9e80 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.248+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df404f100 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.249+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3de401d460 con 0x7f3ddc005080 2024-03-04T12:12:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.249+0000 7f3df0ff9640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 0x7f3de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.249+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3de408a6a0 con 0x7f3ddc005080 2024-03-04T12:12:54.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.249+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 0x7f3de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.250+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 0x7f3de006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3df406d430 tx=0x7f3de8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.254+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3df40c9c00 con 0x7f3ddc005080 2024-03-04T12:12:54.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:12:54.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:12:54.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.463+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3df409a530 con 0x7f3ddc005080 2024-03-04T12:12:54.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.464+0000 7f3df0ff9640 1 -- 172.21.15.148:0/3004386252 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f3de4059120 con 0x7f3ddc005080 2024-03-04T12:12:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:54.466 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 msgr2=0x7f3de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 0x7f3de006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3df406d430 tx=0x7f3de8008040 comp rx=0 tx=0).stop 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 msgr2=0x7f3df4153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3de40011e0 tx=0x7f3de40077a0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 shutdown_connections 2024-03-04T12:12:54.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3de006b4d0 0x7f3de006d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3df41f3bb0 0x7f3df40cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3df41f4680 0x7f3df41f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 --2- 172.21.15.148:0/3004386252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc005080 0x7f3df4153a90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 >> 172.21.15.148:0/3004386252 conn(0x7f3df40b30e0 msgr2=0x7f3df40c7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 shutdown_connections 2024-03-04T12:12:54.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.466+0000 7f3df939d640 1 -- 172.21.15.148:0/3004386252 wait complete. 2024-03-04T12:12:54.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:12:54.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.617+0000 7fc8bef69640 1 Processor -- start 2024-03-04T12:12:54.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.617+0000 7fc8bef69640 1 -- start start 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8b80aead0 0x7fc8b8162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b80b2720 0x7fc8b8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc8b8071fc0 0x7fc8b806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b8162de0 con 0x7fc8b80aead0 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b81630c0 con 0x7fc8b8071fc0 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bef69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81633a0 con 0x7fc8b80b2720 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8b80aead0 0x7fc8b8162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8b80aead0 0x7fc8b8162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40274/0 (socket says 172.21.15.148:40274) 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 -- 172.21.15.148:0/3700248116 learned_addr learned my addr 172.21.15.148:0/3700248116 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8b7fff640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b80b2720 0x7fc8b8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bd506640 1 --2- 172.21.15.148:0/3700248116 >> v2:172.21.15.191:3300/0 conn(0x7fc8b8071fc0 0x7fc8b806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 -- 172.21.15.148:0/3700248116 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b80b2720 msgr2=0x7fc8b8071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b80b2720 0x7fc8b8071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 -- 172.21.15.148:0/3700248116 >> v2:172.21.15.191:3300/0 conn(0x7fc8b8071fc0 msgr2=0x7fc8b806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3700248116 >> v2:172.21.15.191:3300/0 conn(0x7fc8b8071fc0 0x7fc8b806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.618+0000 7fc8bcd05640 1 -- 172.21.15.148:0/3700248116 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8162ad0 con 0x7fc8b80aead0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.619+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b8162590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8166c20 tx=0x7fc8a802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c2b38656293592 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.619+0000 7fc8b77fe640 1 -- 172.21.15.148:0/3700248116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8038070 con 0x7fc8b80aead0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.619+0000 7fc8b77fe640 1 -- 172.21.15.148:0/3700248116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a802f050 con 0x7fc8b80aead0 2024-03-04T12:12:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.619+0000 7fc8b77fe640 1 -- 172.21.15.148:0/3700248116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8031970 con 0x7fc8b80aead0 2024-03-04T12:12:54.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 -- 172.21.15.148:0/3700248116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 msgr2=0x7fc8b8162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b8162590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8166c20 tx=0x7fc8a802b7f0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 -- 172.21.15.148:0/3700248116 shutdown_connections 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b80b2720 0x7fc8b8071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3700248116 >> v2:172.21.15.191:3300/0 conn(0x7fc8b8071fc0 0x7fc8b806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3700248116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b8162590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 -- 172.21.15.148:0/3700248116 >> 172.21.15.148:0/3700248116 conn(0x7fc8b806acd0 msgr2=0x7fc8b806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 -- 172.21.15.148:0/3700248116 shutdown_connections 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.620+0000 7fc8bef69640 1 -- 172.21.15.148:0/3700248116 wait complete. 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.621+0000 7fc8bef69640 1 Processor -- start 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.621+0000 7fc8bef69640 1 -- start start 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b8071fc0 0x7fc8b8147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b80b2720 0x7fc8b80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b806db90 con 0x7fc8b80aead0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b806da10 con 0x7fc8b80b2720 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bef69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b806dd10 con 0x7fc8b8071fc0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40284/0 (socket says 172.21.15.148:40284) 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 -- 172.21.15.148:0/3869893372 learned_addr learned my addr 172.21.15.148:0/3869893372 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 -- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b8071fc0 msgr2=0x7fc8b8147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b8071fc0 0x7fc8b8147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8bd506640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b80b2720 0x7fc8b80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b8071fc0 0x7fc8b8147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 -- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b80b2720 msgr2=0x7fc8b80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b80b2720 0x7fc8b80b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.622+0000 7fc8b7fff640 1 -- 172.21.15.148:0/3869893372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8149c70 con 0x7fc8b80aead0 2024-03-04T12:12:54.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8b7fff640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00d5f0 tx=0x7fc8ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0125f0 con 0x7fc8b80aead0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac019070 con 0x7fc8b80aead0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b814b460 con 0x7fc8b80aead0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0129d0 con 0x7fc8b80aead0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.623+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b814b9e0 con 0x7fc8b80aead0 2024-03-04T12:12:54.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.624+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b8074f60 con 0x7fc8b80aead0 2024-03-04T12:12:54.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.624+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc8ac012b70 con 0x7fc8b80aead0 2024-03-04T12:12:54.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.625+0000 7fc8b57fa640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.625+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.625+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc8ac08aa40 con 0x7fc8b80aead0 2024-03-04T12:12:54.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.625+0000 7fc8bcd05640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8035000 tx=0x7fc8a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.629+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8ac0594c0 con 0x7fc8b80aead0 2024-03-04T12:12:54.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.835+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8b814e1f0 con 0x7fc8b80aead0 2024-03-04T12:12:54.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.835+0000 7fc8b57fa640 1 -- 172.21.15.148:0/3869893372 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fc8ac058dc0 con 0x7fc8b80aead0 2024-03-04T12:12:54.837 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3356958210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356958210},{"type":"v1","addr":"172.21.15.148:6835","nonce":3356958210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1923181538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1923181538},{"type":"v1","addr":"172.21.15.148:6837","nonce":1923181538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1269010581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1269010581},{"type":"v1","addr":"172.21.15.191:6834","nonce":1269010581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:11:54.225836+0000","modified":"2024-03-04T12:12:16.303042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/2190957164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2190957164},{"type":"v1","addr":"172.21.15.191:6835","nonce":2190957164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-04T12:12:54.838 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2024-03-04T12:12:54.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 msgr2=0x7fc8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8035000 tx=0x7fc8a8008030 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 msgr2=0x7fc8b81477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00d5f0 tx=0x7fc8ac00dab0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 shutdown_connections 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b80b2720 0x7fc8b80b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b80aead0 0x7fc8b81477c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 --2- 172.21.15.148:0/3869893372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b8071fc0 0x7fc8b8147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.837+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 >> 172.21.15.148:0/3869893372 conn(0x7fc8b806acd0 msgr2=0x7fc8b8165b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.838+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 shutdown_connections 2024-03-04T12:12:54.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.838+0000 7fc8bef69640 1 -- 172.21.15.148:0/3869893372 wait complete. 2024-03-04T12:12:54.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:12:54.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.951+0000 7fc1483d4640 1 Processor -- start 2024-03-04T12:12:54.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.951+0000 7fc1483d4640 1 -- start start 2024-03-04T12:12:54.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc1483d4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc14006acd0 0x7fc14015c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc1483d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc1400cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc1483d4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1400cf700 con 0x7fc14006b0b0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc1483d4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1400d02d0 con 0x7fc14006acd0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc1483d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1400cf9a0 con 0x7fc1400cc330 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc14596f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc14006b0b0 0x7fc14006b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59404/0 (socket says 172.21.15.148:59404) 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc14596f640 1 -- 172.21.15.148:0/283166917 learned_addr learned my addr 172.21.15.148:0/283166917 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc146971640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc1400cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc146170640 1 --2- 172.21.15.148:0/283166917 >> v2:172.21.15.191:3300/0 conn(0x7fc14006acd0 0x7fc14015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088711047 0 0) 0x7fc1400cf700 con 0x7fc14006b0b0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.952+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc12c003520 con 0x7fc14006b0b0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613605644 0 0) 0x7fc12c003520 con 0x7fc14006b0b0 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 msgr2=0x7fc1400cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc1400cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 >> v2:172.21.15.191:3300/0 conn(0x7fc14006acd0 msgr2=0x7fc14015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 --2- 172.21.15.148:0/283166917 >> v2:172.21.15.191:3300/0 conn(0x7fc14006acd0 0x7fc14015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1400cfc80 con 0x7fc14006b0b0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fc130001670 con 0x7fc14006b0b0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 >> v1:172.21.15.148:6789/0 conn(0x7fc14006b0b0 legacy=0x7fc14006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc12c0039b0 0x7fc12c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc12c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc12c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1400cf9a0 con 0x7fc12c004800 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00b9c0 con 0x7fc12c0039b0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00bb70 con 0x7fc1400cc330 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc12c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc146170640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc12c0039b0 0x7fc12c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc146971640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc12c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 -- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 msgr2=0x7fc12c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1400cc330 0x7fc12c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 -- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc12c0039b0 msgr2=0x7fc12c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc12c0039b0 0x7fc12c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.953+0000 7fc14596f640 1 -- 172.21.15.148:0/283166917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c003520 con 0x7fc12c004800 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.954+0000 7fc14596f640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc12c004c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc130001050 tx=0x7fc12c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5161705e62e24cc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.954+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130007d70 con 0x7fc12c004800 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.954+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130030070 con 0x7fc12c004800 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.954+0000 7fc14516e640 1 -- 172.21.15.148:0/283166917 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130005d30 con 0x7fc12c004800 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 -- 172.21.15.148:0/283166917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 msgr2=0x7fc12c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 --2- 172.21.15.148:0/283166917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc12c004c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc130001050 tx=0x7fc12c005830 comp rx=0 tx=0).stop 2024-03-04T12:12:54.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc146971640 1 -- 172.21.15.148:0/283166917 reap_dead start 2024-03-04T12:12:54.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 -- 172.21.15.148:0/283166917 shutdown_connections 2024-03-04T12:12:54.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 -- 172.21.15.148:0/283166917 >> 172.21.15.148:0/283166917 conn(0x7fc1400d0770 msgr2=0x7fc1400d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:54.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 -- 172.21.15.148:0/283166917 shutdown_connections 2024-03-04T12:12:54.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.955+0000 7fc1483d4640 1 -- 172.21.15.148:0/283166917 wait complete. 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.956+0000 7fc1483d4640 1 Processor -- start 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.956+0000 7fc1483d4640 1 -- start start 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.956+0000 7fc1483d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.956+0000 7fc1483d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 0x7fc1400ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc140147c60 0x7fc1400cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140159850 con 0x7fc12c004800 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1400d13e0 con 0x7fc1401477e0 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401599d0 con 0x7fc140147c60 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40296/0 (socket says 172.21.15.148:40296) 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 -- 172.21.15.148:0/1619871265 learned_addr learned my addr 172.21.15.148:0/1619871265 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc14596f640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 0x7fc1400ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146971640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc140147c60 0x7fc1400cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 -- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc140147c60 msgr2=0x7fc1400cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc140147c60 0x7fc1400cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 -- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 msgr2=0x7fc1400ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 0x7fc1400ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 -- 172.21.15.148:0/1619871265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1401f0fc0 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc14596f640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 0x7fc1400ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc146170640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0118e0 tx=0x7fc13c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00faf0 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c00fc90 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1400b4940 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c014440 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.957+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1400b4be0 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.958+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc140074f60 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.959+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc13c012040 con 0x7fc12c004800 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.959+0000 7fc136ffd640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 0x7fc12406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:54.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.959+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc13c08ad30 con 0x7fc12c004800 2024-03-04T12:12:54.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.959+0000 7fc14596f640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 0x7fc12406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:54.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.959+0000 7fc14596f640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 0x7fc12406d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc130034000 tx=0x7fc130007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:54.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:54.963+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1401f0fc0 con 0x7fc12c004800 2024-03-04T12:12:55.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.135+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc14015e3e0 con 0x7fc12c004800 2024-03-04T12:12:55.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.136+0000 7fc136ffd640 1 -- 172.21.15.148:0/1619871265 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fc13c019030 con 0x7fc12c004800 2024-03-04T12:12:55.138 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:12:55.138 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:12:36.014075+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:11:52.775816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T12:11:53.193920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T12:11:56.927475+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:12:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 msgr2=0x7fc12406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 0x7fc12406d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc130034000 tx=0x7fc130007000 comp rx=0 tx=0).stop 2024-03-04T12:12:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 msgr2=0x7fc1401f0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0118e0 tx=0x7fc13c011da0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 shutdown_connections 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fc12406b500 0x7fc12406d9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc140147c60 0x7fc1400cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1401477e0 0x7fc1400ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 --2- 172.21.15.148:0/1619871265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c004800 0x7fc1401f0a80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 >> 172.21.15.148:0/1619871265 conn(0x7fc1400d0770 msgr2=0x7fc1400b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 shutdown_connections 2024-03-04T12:12:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.138+0000 7fc1483d4640 1 -- 172.21.15.148:0/1619871265 wait complete. 2024-03-04T12:12:55.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-04T12:12:55.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.251+0000 7fcd20836640 1 Processor -- start 2024-03-04T12:12:55.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.251+0000 7fcd20836640 1 -- start start 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd20836640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd1c15d660 0x7fcd1c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd20836640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c15da40 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd20836640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c15ebb0 con 0x7fcd1c071a60 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd20836640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c15cbc0 con 0x7fcd1c15d660 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd13fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcd1c0aeb20 0x7fcd1c071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59416/0 (socket says 172.21.15.148:59416) 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.252+0000 7fcd13fff640 1 -- 172.21.15.148:0/3596002149 learned_addr learned my addr 172.21.15.148:0/3596002149 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831516634 0 0) 0x7fcd1c15da40 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd08004630 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1a59c640 1 --2- 172.21.15.148:0/3596002149 >> v2:172.21.15.191:3300/0 conn(0x7fcd1c15d660 0x7fcd1c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 374091415 0 0) 0x7fcd08004630 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 >> v2:172.21.15.191:3300/0 conn(0x7fcd1c15d660 msgr2=0x7fcd1c15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 --2- 172.21.15.148:0/3596002149 >> v2:172.21.15.191:3300/0 conn(0x7fcd1c15d660 0x7fcd1c15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 >> v1:172.21.15.191:6789/0 conn(0x7fcd1c071a60 legacy=0x7fcd1c150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0af090 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fcd04001690 con 0x7fcd1c0aeb20 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 >> v1:172.21.15.148:6789/0 conn(0x7fcd1c0aeb20 legacy=0x7fcd1c071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd08004ac0 0x7fcd08004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd08005910 0x7fcd08005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 0x7fcd0800cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15cbc0 con 0x7fcd080062d0 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0800d270 con 0x7fcd08005910 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0800d470 con 0x7fcd08004ac0 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 0x7fcd0800cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd13fff640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd08005910 0x7fcd08005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1a59c640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd08004ac0 0x7fcd08004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd08004ac0 msgr2=0x7fcd08004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd08004ac0 0x7fcd08004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd08005910 msgr2=0x7fcd08005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd08005910 0x7fcd08005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.253+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/3596002149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08004630 con 0x7fcd080062d0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.254+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/3596002149 reap_dead start 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.254+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 0x7fcd0800cad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001e50 tx=0x7fcd08006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb227b79fd5c506 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.254+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14001a70 con 0x7fcd080062d0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.254+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14003bb0 con 0x7fcd080062d0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.254+0000 7fcd19d9b640 1 -- 172.21.15.148:0/3596002149 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140050d0 con 0x7fcd080062d0 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 -- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 msgr2=0x7fcd0800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 0x7fcd0800cad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001e50 tx=0x7fcd08006940 comp rx=0 tx=0).stop 2024-03-04T12:12:55.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 -- 172.21.15.148:0/3596002149 shutdown_connections 2024-03-04T12:12:55.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 --2- 172.21.15.148:0/3596002149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd080062d0 0x7fcd0800cad0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 -- 172.21.15.148:0/3596002149 >> 172.21.15.148:0/3596002149 conn(0x7fcd1c06acd0 msgr2=0x7fcd1c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:55.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 -- 172.21.15.148:0/3596002149 shutdown_connections 2024-03-04T12:12:55.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.255+0000 7fcd20836640 1 -- 172.21.15.148:0/3596002149 wait complete. 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.256+0000 7fcd20836640 1 Processor -- start 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.256+0000 7fcd20836640 1 -- start start 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd080062d0 0x7fcd1c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c149030 0x7fcd1c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15afb0 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15acd0 con 0x7fcd080062d0 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd20836640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15a660 con 0x7fcd1c149030 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40316/0 (socket says 172.21.15.148:40316) 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/2405052040 learned_addr learned my addr 172.21.15.148:0/2405052040 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd13fff640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c149030 0x7fcd1c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1a59c640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd080062d0 0x7fcd1c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c149030 msgr2=0x7fcd1c148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c149030 0x7fcd1c148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd080062d0 msgr2=0x7fcd1c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd080062d0 0x7fcd1c1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 -- 172.21.15.148:0/2405052040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c154b10 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.257+0000 7fcd1ad9d640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402c060 tx=0x7fcd140014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.258+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14006d60 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.258+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14039480 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.258+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14039830 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.258+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c0cfbd0 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.258+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c0b08f0 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.259+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c074f60 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.259+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd140399d0 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.259+0000 7fcd12ffd640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 0x7fccfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.260+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcd140ac2a0 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.260+0000 7fcd1a59c640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 0x7fccfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.260+0000 7fcd1a59c640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 0x7fccfc06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009a80 tx=0x7fcd0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.264+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd1c154b10 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.471+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcd1c072460 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.473 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-03-04T12:12:55.471+0000 7fd7da450640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T12:12:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.473+0000 7fcd12ffd640 1 -- 172.21.15.148:0/2405052040 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd1403e070 con 0x7fcd1c0b3c30 2024-03-04T12:12:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-04T12:12:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.475+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 msgr2=0x7fccfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.475+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 0x7fccfc06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009a80 tx=0x7fcd0c008040 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.475+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 msgr2=0x7fcd1c0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.475+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402c060 tx=0x7fcd140014a0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 shutdown_connections 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fccfc06b500 0x7fccfc06d9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd1c0b3c30 0x7fcd1c0b40b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c149030 0x7fcd1c148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 --2- 172.21.15.148:0/2405052040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd080062d0 0x7fcd1c1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 >> 172.21.15.148:0/2405052040 conn(0x7fcd1c06acd0 msgr2=0x7fcd1c0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 shutdown_connections 2024-03-04T12:12:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.476+0000 7fcd20836640 1 -- 172.21.15.148:0/2405052040 wait complete. 2024-03-04T12:12:55.481 INFO:tasks.ceph.mds.b.smithi191.stderr: -4966> 2024-03-04T12:11:59.392+0000 7f453a2a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T12:12:55.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-04T12:12:55.513 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-04T12:12:55.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.589+0000 7f7059d45640 1 Processor -- start 2024-03-04T12:12:55.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 -- start start 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f705415df00 0x7f705406b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f70540cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70540701d0 con 0x7f705406fa80 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70540704b0 con 0x7f705415df00 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.590+0000 7f7059d45640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cc8e0 con 0x7f70540cfa40 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f70537fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f705415df00 0x7f705406b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f7053fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f70540cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f704affd640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f705406fa80 0x7f70540cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:44826/0 (socket says 172.21.15.148:44826) 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f704affd640 1 -- 172.21.15.148:0/1948726520 learned_addr learned my addr 172.21.15.148:0/1948726520 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f70537fe640 1 -- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 msgr2=0x7f70540cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f70537fe640 1 --2- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f70540cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f70537fe640 1 -- 172.21.15.148:0/1948726520 >> v1:172.21.15.191:6789/0 conn(0x7f705406fa80 legacy=0x7f70540cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f70537fe640 1 -- 172.21.15.148:0/1948726520 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540ccbc0 con 0x7f705415df00 2024-03-04T12:12:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.591+0000 7f7052ffd640 1 -- 172.21.15.148:0/1948726520 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312988868 0 0) 0x7f70540701d0 con 0x7f705406fa80 2024-03-04T12:12:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.592+0000 7f70537fe640 1 --2- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f705406b700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70440097a0 tx=0x7f704402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d679df62dc8da81b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.592+0000 7f7052ffd640 1 -- 172.21.15.148:0/1948726520 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044035020 con 0x7f705415df00 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.592+0000 7f7052ffd640 1 -- 172.21.15.148:0/1948726520 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044031830 con 0x7f705415df00 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.592+0000 7f7052ffd640 1 -- 172.21.15.148:0/1948726520 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044031c00 con 0x7f705415df00 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 -- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 msgr2=0x7f705406b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 --2- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f705406b700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70440097a0 tx=0x7f704402b8d0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 -- 172.21.15.148:0/1948726520 shutdown_connections 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 --2- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f70540cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 --2- 172.21.15.148:0/1948726520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f705406b700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 -- 172.21.15.148:0/1948726520 >> 172.21.15.148:0/1948726520 conn(0x7f70540b2250 msgr2=0x7f70540b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 -- 172.21.15.148:0/1948726520 shutdown_connections 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.593+0000 7f7059d45640 1 -- 172.21.15.148:0/1948726520 wait complete. 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.594+0000 7f7059d45640 1 Processor -- start 2024-03-04T12:12:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.594+0000 7f7059d45640 1 -- start start 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f7054148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f70540b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540d0410 con 0x7f705406fa80 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705406fee0 con 0x7f705415df00 2024-03-04T12:12:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7059d45640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cd3c0 con 0x7f70540cfa40 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40318/0 (socket says 172.21.15.148:40318) 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 -- 172.21.15.148:0/1605866583 learned_addr learned my addr 172.21.15.148:0/1605866583 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f704affd640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f7054148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f7053fff640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f70540b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 -- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 msgr2=0x7f7054148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f7054148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 -- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 msgr2=0x7f70540b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f70540b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 -- 172.21.15.148:0/1605866583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705414aff0 con 0x7f705406fa80 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.595+0000 7f70537fe640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7044034040 tx=0x7f70440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.596+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044040070 con 0x7f705406fa80 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.596+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704403a910 con 0x7f705406fa80 2024-03-04T12:12:55.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.596+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705414b2d0 con 0x7f705406fa80 2024-03-04T12:12:55.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.596+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044035040 con 0x7f705406fa80 2024-03-04T12:12:55.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.596+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705414b890 con 0x7f705406fa80 2024-03-04T12:12:55.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.597+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054074f60 con 0x7f705406fa80 2024-03-04T12:12:55.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.597+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7044038020 con 0x7f705406fa80 2024-03-04T12:12:55.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.597+0000 7f7050ff9640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 0x7f703406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:55.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.597+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f70440abee0 con 0x7f705406fa80 2024-03-04T12:12:55.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.597+0000 7f704affd640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 0x7f703406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:55.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.598+0000 7f704affd640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 0x7f703406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f703c000f60 tx=0x7f703c007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.602+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7044031800 con 0x7f705406fa80 2024-03-04T12:12:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:55.808+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7054148c80 con 0x7f705406fa80 2024-03-04T12:12:56.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.474+0000 7f7050ff9640 1 -- 172.21.15.148:0/1605866583 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7054148c80 con 0x7f705406fa80 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 msgr2=0x7f703406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 0x7f703406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f703c000f60 tx=0x7f703c007d50 comp rx=0 tx=0).stop 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 msgr2=0x7f70540af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7044034040 tx=0x7f70440046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 shutdown_connections 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f703406b4d0 0x7f703406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f705415df00 0x7f70540b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70540cfa40 0x7f7054148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 --2- 172.21.15.148:0/1605866583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f705406fa80 0x7f70540af620 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 >> 172.21.15.148:0/1605866583 conn(0x7f70540b2250 msgr2=0x7f70540cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:56.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 shutdown_connections 2024-03-04T12:12:56.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.476+0000 7f7059d45640 1 -- 172.21.15.148:0/1605866583 wait complete. 2024-03-04T12:12:56.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-04T12:12:56.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.595+0000 7fbc9d73b640 1 Processor -- start 2024-03-04T12:12:56.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.595+0000 7fbc9d73b640 1 -- start start 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc9d73b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbc9815e3c0 0x7fbc980d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc9d73b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9806cee0 con 0x7fbc980cf820 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc9d73b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9806d1c0 con 0x7fbc9815dfe0 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc9d73b640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9806daf0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc977fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbc980cf820 0x7fbc980cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59430/0 (socket says 172.21.15.148:59430) 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc977fe640 1 -- 172.21.15.148:0/2328013396 learned_addr learned my addr 172.21.15.148:0/2328013396 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.596+0000 7fbc967fc640 1 --2- 172.21.15.148:0/2328013396 >> v2:172.21.15.191:3300/0 conn(0x7fbc9815e3c0 0x7fbc980d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc95ffb640 1 -- 172.21.15.148:0/2328013396 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912526029 0 0) 0x7fbc9806cee0 con 0x7fbc980cf820 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc95ffb640 1 -- 172.21.15.148:0/2328013396 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc80003520 con 0x7fbc980cf820 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc967fc640 1 -- 172.21.15.148:0/2328013396 >> v1:172.21.15.191:6789/0 conn(0x7fbc9815dfe0 legacy=0x7fbc9806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc967fc640 1 -- 172.21.15.148:0/2328013396 >> v1:172.21.15.148:6789/0 conn(0x7fbc980cf820 legacy=0x7fbc980cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc967fc640 1 -- 172.21.15.148:0/2328013396 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9806d4a0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.597+0000 7fbc967fc640 1 --2- 172.21.15.148:0/2328013396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9815e3c0 0x7fbc980d17d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0097a0 tx=0x7fbc8c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380461866f2cc123 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc95ffb640 1 -- 172.21.15.148:0/2328013396 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c034070 con 0x7fbc9815e3c0 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc95ffb640 1 -- 172.21.15.148:0/2328013396 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c030610 con 0x7fbc9815e3c0 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc95ffb640 1 -- 172.21.15.148:0/2328013396 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0309e0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 -- 172.21.15.148:0/2328013396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9815e3c0 msgr2=0x7fbc980d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/2328013396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9815e3c0 0x7fbc980d17d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0097a0 tx=0x7fbc8c02b960 comp rx=0 tx=0).stop 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 -- 172.21.15.148:0/2328013396 shutdown_connections 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/2328013396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9815e3c0 0x7fbc980d17d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 -- 172.21.15.148:0/2328013396 >> 172.21.15.148:0/2328013396 conn(0x7fbc980b34f0 msgr2=0x7fbc980b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 -- 172.21.15.148:0/2328013396 shutdown_connections 2024-03-04T12:12:56.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.598+0000 7fbc9d73b640 1 -- 172.21.15.148:0/2328013396 wait complete. 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 Processor -- start 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 -- start start 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 0x7fbc980c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc9815dfe0 0x7fbc98148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980d1e90 con 0x7fbc9815e3c0 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980d1d10 con 0x7fbc980cf820 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc9d73b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980d2010 con 0x7fbc9815dfe0 2024-03-04T12:12:56.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.600+0000 7fbc977fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40330/0 (socket says 172.21.15.148:40330) 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 -- 172.21.15.148:0/3394009793 learned_addr learned my addr 172.21.15.148:0/3394009793 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc96ffd640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 0x7fbc980c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc967fc640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc9815dfe0 0x7fbc98148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 -- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc9815dfe0 msgr2=0x7fbc98148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc9815dfe0 0x7fbc98148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 -- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 msgr2=0x7fbc980c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 0x7fbc980c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 -- 172.21.15.148:0/3394009793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980c5530 con 0x7fbc9815e3c0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc96ffd640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 0x7fbc980c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc977fe640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c400 tx=0x7fbc9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90017070 con 0x7fbc9815e3c0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc90005ce0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900072d0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc980c5810 con 0x7fbc9815e3c0 2024-03-04T12:12:56.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.601+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980c7ff0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.602+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98074f60 con 0x7fbc9815e3c0 2024-03-04T12:12:56.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.603+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbc9001a070 con 0x7fbc9815e3c0 2024-03-04T12:12:56.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.603+0000 7fbc7f7fe640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:56.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.603+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fbc900898d0 con 0x7fbc9815e3c0 2024-03-04T12:12:56.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.603+0000 7fbc96ffd640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:56.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.603+0000 7fbc96ffd640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 0x7fbc8406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cd250 tx=0x7fbc88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:56.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.607+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc90057d30 con 0x7fbc9815e3c0 2024-03-04T12:12:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:56.781+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc98160b00 con 0x7fbc9815e3c0 2024-03-04T12:12:57.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.475+0000 7fbc7f7fe640 1 -- 172.21.15.148:0/3394009793 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbc9005bdd0 con 0x7fbc9815e3c0 2024-03-04T12:12:57.477 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-03-04T12:12:57.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.478+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 msgr2=0x7fbc8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:57.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.478+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 0x7fbc8406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cd250 tx=0x7fbc88009040 comp rx=0 tx=0).stop 2024-03-04T12:12:57.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.478+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 msgr2=0x7fbc981f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.478+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c400 tx=0x7fbc9000c8c0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.478+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 shutdown_connections 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9815e3c0 0x7fbc981f7680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc9815dfe0 0x7fbc98148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 --2- 172.21.15.148:0/3394009793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980cf820 0x7fbc980c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 >> 172.21.15.148:0/3394009793 conn(0x7fbc980b34f0 msgr2=0x7fbc980cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 shutdown_connections 2024-03-04T12:12:57.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.479+0000 7fbc9d73b640 1 -- 172.21.15.148:0/3394009793 wait complete. 2024-03-04T12:12:57.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-04T12:12:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 Processor -- start 2024-03-04T12:12:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 -- start start 2024-03-04T12:12:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f30180700e0 0x7f301806acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f301806f1b0 0x7f30180cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301806b790 con 0x7f301806b210 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301806e6d0 con 0x7f301806f1b0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f301cacd640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301806e9b0 con 0x7f30180700e0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.598+0000 7f300ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f301806f1b0 0x7f30180cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f301806f1b0 0x7f30180cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40338/0 (socket says 172.21.15.148:40338) 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 -- 172.21.15.148:0/2188459570 learned_addr learned my addr 172.21.15.148:0/2188459570 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f301659c640 1 --2- 172.21.15.148:0/2188459570 >> v2:172.21.15.191:3300/0 conn(0x7f30180700e0 0x7f301806acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 -- 172.21.15.148:0/2188459570 >> v2:172.21.15.191:3300/0 conn(0x7f30180700e0 msgr2=0x7f301806acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 --2- 172.21.15.148:0/2188459570 >> v2:172.21.15.191:3300/0 conn(0x7f30180700e0 0x7f301806acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 -- 172.21.15.148:0/2188459570 >> v1:172.21.15.191:6789/0 conn(0x7f301806b210 legacy=0x7f301815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.599+0000 7f300ffff640 1 -- 172.21.15.148:0/2188459570 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301806ec90 con 0x7f301806f1b0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f300ffff640 1 --2- 172.21.15.148:0/2188459570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180cf9b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3018072440 tx=0x7f300002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc5f61c1c52b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f3015d9b640 1 -- 172.21.15.148:0/2188459570 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000035020 con 0x7f301806f1b0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f3015d9b640 1 -- 172.21.15.148:0/2188459570 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000031870 con 0x7f301806f1b0 2024-03-04T12:12:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f3015d9b640 1 -- 172.21.15.148:0/2188459570 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000031c40 con 0x7f301806f1b0 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 -- 172.21.15.148:0/2188459570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 msgr2=0x7f30180cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 --2- 172.21.15.148:0/2188459570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180cf9b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3018072440 tx=0x7f300002b910 comp rx=0 tx=0).stop 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 -- 172.21.15.148:0/2188459570 shutdown_connections 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 --2- 172.21.15.148:0/2188459570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180cf9b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 --2- 172.21.15.148:0/2188459570 >> v2:172.21.15.191:3300/0 conn(0x7f30180700e0 0x7f301806acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 -- 172.21.15.148:0/2188459570 >> 172.21.15.148:0/2188459570 conn(0x7f3018099a00 msgr2=0x7f30180b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 -- 172.21.15.148:0/2188459570 shutdown_connections 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.600+0000 7f301cacd640 1 -- 172.21.15.148:0/2188459570 wait complete. 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 Processor -- start 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 -- start start 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f301806b210 0x7f30181f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30180700e0 0x7f30180b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180cfef0 con 0x7f301806f1b0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180704c0 con 0x7f301806b210 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301cacd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30180d0070 con 0x7f30180700e0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f300ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f300ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40342/0 (socket says 172.21.15.148:40342) 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f300ffff640 1 -- 172.21.15.148:0/2601497385 learned_addr learned my addr 172.21.15.148:0/2601497385 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.602+0000 7f301659c640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f301806b210 0x7f30181f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f3016d9d640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30180700e0 0x7f30180b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 -- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30180700e0 msgr2=0x7f30180b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30180700e0 0x7f30180b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 -- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f301806b210 msgr2=0x7f30181f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f301806b210 0x7f30181f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 -- 172.21.15.148:0/2601497385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180b3f50 con 0x7f301806f1b0 2024-03-04T12:12:57.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300ffff640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f30000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000040070 con 0x7f301806f1b0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000030d00 con 0x7f301806f1b0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30180b4230 con 0x7f301806f1b0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000035040 con 0x7f301806f1b0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.603+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30180b4860 con 0x7f301806f1b0 2024-03-04T12:12:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.604+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3018074f60 con 0x7f301806f1b0 2024-03-04T12:12:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.605+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3000038020 con 0x7f301806f1b0 2024-03-04T12:12:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.605+0000 7f300effd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.605+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f30000ac1b0 con 0x7f301806f1b0 2024-03-04T12:12:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.605+0000 7f301659c640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.605+0000 7f301659c640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30080079a0 tx=0x7f3008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:57.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.609+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f300007a7d0 con 0x7f301806f1b0 2024-03-04T12:12:57.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:57.781+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30180cc270 con 0x7f301806f1b0 2024-03-04T12:12:58.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.488+0000 7f300effd640 1 -- 172.21.15.148:0/2601497385 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f300007e870 con 0x7f301806f1b0 2024-03-04T12:12:58.490 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-03-04T12:12:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 msgr2=0x7f2ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30080079a0 tx=0x7f3008008040 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 msgr2=0x7f30180b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f30000046c0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 shutdown_connections 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30180700e0 0x7f30180b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301806f1b0 0x7f30180b1160 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 --2- 172.21.15.148:0/2601497385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f301806b210 0x7f30181f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 >> 172.21.15.148:0/2601497385 conn(0x7f3018099a00 msgr2=0x7f30181e6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 shutdown_connections 2024-03-04T12:12:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.491+0000 7f301cacd640 1 -- 172.21.15.148:0/2601497385 wait complete. 2024-03-04T12:12:58.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-04T12:12:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.621+0000 7f6dc9329640 1 Processor -- start 2024-03-04T12:12:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.621+0000 7f6dc9329640 1 -- start start 2024-03-04T12:12:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc9329640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6dc415db20 0x7f6dc40b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc9329640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc40ce120 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc9329640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc40ced90 con 0x7f6dc40d3c20 2024-03-04T12:12:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc9329640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc40ce730 con 0x7f6dc415db20 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6dc40b5bc0 0x7f6dc40b6b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59442/0 (socket says 172.21.15.148:59442) 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc37fe640 1 -- 172.21.15.148:0/3981746076 learned_addr learned my addr 172.21.15.148:0/3981746076 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34768881 0 0) 0x7f6dc40ce120 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dac003520 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc2ffd640 1 --2- 172.21.15.148:0/3981746076 >> v2:172.21.15.191:3300/0 conn(0x7f6dc415db20 0x7f6dc40b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.622+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1407822934 0 0) 0x7f6dac003520 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 >> v2:172.21.15.191:3300/0 conn(0x7f6dc415db20 msgr2=0x7f6dc40b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 --2- 172.21.15.148:0/3981746076 >> v2:172.21.15.191:3300/0 conn(0x7f6dc415db20 0x7f6dc40b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 >> v1:172.21.15.191:6789/0 conn(0x7f6dc40d3c20 legacy=0x7f6dc40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc40ce3c0 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f6dbc001670 con 0x7f6dc40b5bc0 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 >> v1:172.21.15.148:6789/0 conn(0x7f6dc40b5bc0 legacy=0x7f6dc40b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dac0039b0 0x7f6dac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac004800 0x7f6dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 0x7f6dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40ce730 con 0x7f6dac0051c0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac00c070 con 0x7f6dac0039b0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac00c270 con 0x7f6dac004800 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 0x7f6dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc2ffd640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dac0039b0 0x7f6dac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc27fc640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac004800 0x7f6dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 -- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac004800 msgr2=0x7f6dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac004800 0x7f6dac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 -- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dac0039b0 msgr2=0x7f6dac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dac0039b0 0x7f6dac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.623+0000 7f6dc37fe640 1 -- 172.21.15.148:0/3981746076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac003520 con 0x7f6dac0051c0 2024-03-04T12:12:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc37fe640 1 -- 172.21.15.148:0/3981746076 reap_dead start 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc37fe640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 0x7f6dac00b9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40d49b0 tx=0x7f6dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798ccd2dafb39e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc007ce0 con 0x7f6dac0051c0 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc030070 con 0x7f6dac0051c0 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc1ffb640 1 -- 172.21.15.148:0/3981746076 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc005d50 con 0x7f6dac0051c0 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc9329640 1 -- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 msgr2=0x7f6dac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.624+0000 7f6dc9329640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 0x7f6dac00b9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40d49b0 tx=0x7f6dac005830 comp rx=0 tx=0).stop 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.625+0000 7f6dc9329640 1 -- 172.21.15.148:0/3981746076 shutdown_connections 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.625+0000 7f6dc9329640 1 --2- 172.21.15.148:0/3981746076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dac0051c0 0x7f6dac00b9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.625+0000 7f6dc9329640 1 -- 172.21.15.148:0/3981746076 >> 172.21.15.148:0/3981746076 conn(0x7f6dc4072820 msgr2=0x7f6dc4072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:58.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.625+0000 7f6dc9329640 1 -- 172.21.15.148:0/3981746076 shutdown_connections 2024-03-04T12:12:58.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.625+0000 7f6dc9329640 1 -- 172.21.15.148:0/3981746076 wait complete. 2024-03-04T12:12:58.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.626+0000 7f6dc9329640 1 Processor -- start 2024-03-04T12:12:58.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.626+0000 7f6dc9329640 1 -- start start 2024-03-04T12:12:58.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac0051c0 0x7f6dc4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4155d80 0x7f6dc4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415df00 con 0x7f6dc4149750 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40b6300 con 0x7f6dc4155d80 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415e080 con 0x7f6dac0051c0 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40358/0 (socket says 172.21.15.148:40358) 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 -- 172.21.15.148:0/2932500612 learned_addr learned my addr 172.21.15.148:0/2932500612 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc2ffd640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac0051c0 0x7f6dc4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc37fe640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4155d80 0x7f6dc4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 -- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac0051c0 msgr2=0x7f6dc4153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac0051c0 0x7f6dc4153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 -- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4155d80 msgr2=0x7f6dc4156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:58.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4155d80 0x7f6dc4156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 -- 172.21.15.148:0/2932500612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc40cb200 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc27fc640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6db800f540 tx=0x7f6db800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db80052e0 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8018070 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc414a110 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8003830 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.627+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc40cb750 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.629+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc4074f60 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.629+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6db80039d0 con 0x7f6dc4149750 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.630+0000 7f6dab7fe640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 0x7f6db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:58.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.630+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6db8089fd0 con 0x7f6dc4149750 2024-03-04T12:12:58.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.631+0000 7f6dc2ffd640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 0x7f6db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:58.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.632+0000 7f6dc2ffd640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 0x7f6db006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009fd0 tx=0x7f6db400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:58.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.636+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6db8058dc0 con 0x7f6dc4149750 2024-03-04T12:12:58.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:58.815+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6dc415c7f0 con 0x7f6dc4149750 2024-03-04T12:12:59.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.493+0000 7f6dab7fe640 1 -- 172.21.15.148:0/2932500612 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6db80586c0 con 0x7f6dc4149750 2024-03-04T12:12:59.495 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' removed 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 msgr2=0x7f6db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 0x7f6db006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009fd0 tx=0x7f6db400b040 comp rx=0 tx=0).stop 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 msgr2=0x7f6dc4149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6db800f540 tx=0x7f6db800fa00 comp rx=0 tx=0).stop 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 shutdown_connections 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f6db006b4d0 0x7f6db006d990 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009fd0 tx=0x7f6db400b040 comp rx=0 tx=0).stop 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4155d80 0x7f6dc4156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc4149750 0x7f6dc4149bd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 --2- 172.21.15.148:0/2932500612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dac0051c0 0x7f6dc4153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 >> 172.21.15.148:0/2932500612 conn(0x7f6dc4072820 msgr2=0x7f6dc40d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 shutdown_connections 2024-03-04T12:12:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.497+0000 7f6dc9329640 1 -- 172.21.15.148:0/2932500612 wait complete. 2024-03-04T12:12:59.513 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T12:12:59.513 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-04T12:12:59.513 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T12:12:59.513 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T12:12:59.514 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T12:12:59.514 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-04T12:12:59.514 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T12:12:59.514 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T12:12:59.514 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T12:12:59.514 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-04T12:12:59.514 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T12:12:59.515 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T12:12:59.515 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-04T12:12:59.515 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-04T12:12:59.515 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-04T12:12:59.515 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T12:12:59.525 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T12:12:59.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T12:12:59.526 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T12:12:59.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T12:12:59.527 INFO:tasks.ceph.mds.d:Stopped 2024-03-04T12:12:59.527 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-04T12:12:59.528 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T12:12:59.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T12:12:59.568 INFO:tasks.ceph.mds.a:Started 2024-03-04T12:12:59.568 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:59.569 INFO:tasks.ceph.mds.c:Started 2024-03-04T12:12:59.570 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:59.571 INFO:tasks.ceph.mds.d:Started 2024-03-04T12:12:59.571 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:59.571 INFO:tasks.ceph.mds.b:Started 2024-03-04T12:12:59.571 DEBUG:teuthology.parallel:result is None 2024-03-04T12:12:59.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-04T12:12:59.648 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-04T12:12:59.649 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-03-04T12:12:59.654 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-04T12:12:59.654 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-04T12:12:59.654 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-04T12:12:59.653+0000 7fea1f8a3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-04T12:12:59.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.682+0000 7ff517c61640 1 Processor -- start 2024-03-04T12:12:59.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.682+0000 7ff517c61640 1 -- start start 2024-03-04T12:12:59.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff517c61640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff5100cf570 0x7ff5100cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff517c61640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff5100cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff517c61640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51015cf30 con 0x7ff5100cfa30 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff517c61640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51015caa0 con 0x7ff5100cf570 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff517c61640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51015c170 con 0x7ff51015bc40 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5159fd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff5100cfa30 0x7ff5100cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59450/0 (socket says 172.21.15.148:59450) 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5159fd640 1 -- 172.21.15.148:0/2439032815 learned_addr learned my addr 172.21.15.148:0/2439032815 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5151fc640 1 --2- 172.21.15.148:0/2439032815 >> v2:172.21.15.191:3300/0 conn(0x7ff5100cf570 0x7ff5100cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620160960 0 0) 0x7ff51015cf30 con 0x7ff5100cfa30 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5161fe640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff5100cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4fc003520 con 0x7ff5100cfa30 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2464811417 0 0) 0x7ff4fc003520 con 0x7ff5100cfa30 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 msgr2=0x7ff5100cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff5100cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 >> v2:172.21.15.191:3300/0 conn(0x7ff5100cf570 msgr2=0x7ff5100cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.683+0000 7ff5149fb640 1 --2- 172.21.15.148:0/2439032815 >> v2:172.21.15.191:3300/0 conn(0x7ff5100cf570 0x7ff5100cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51015c450 con 0x7ff5100cfa30 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7ff50c001670 con 0x7ff5100cfa30 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5151fc640 1 --2- 172.21.15.148:0/2439032815 >> v2:172.21.15.191:3300/0 conn(0x7ff5100cf570 0x7ff5100cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 >> v1:172.21.15.148:6789/0 conn(0x7ff5100cfa30 legacy=0x7ff5100cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0039b0 0x7ff4fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4fc004800 0x7ff4fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff4fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51015caa0 con 0x7ff4fc0039b0 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51015c170 con 0x7ff4fc004800 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc00fdd0 con 0x7ff51015bc40 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0039b0 0x7ff4fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5151fc640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4fc004800 0x7ff4fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5161fe640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff4fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 -- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 msgr2=0x7ff4fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff51015bc40 0x7ff4fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 -- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4fc004800 msgr2=0x7ff4fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4fc004800 0x7ff4fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.684+0000 7ff5159fd640 1 -- 172.21.15.148:0/2439032815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc003520 con 0x7ff4fc0039b0 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff5159fd640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0039b0 0x7ff4fc003dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001050 tx=0x7ff4fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9489a8316b617 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c007d70 con 0x7ff4fc0039b0 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c030070 con 0x7ff4fc0039b0 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff5149fb640 1 -- 172.21.15.148:0/2439032815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c02cd80 con 0x7ff4fc0039b0 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff517c61640 1 -- 172.21.15.148:0/2439032815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0039b0 msgr2=0x7ff4fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff517c61640 1 --2- 172.21.15.148:0/2439032815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0039b0 0x7ff4fc003dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001050 tx=0x7ff4fc005830 comp rx=0 tx=0).stop 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.685+0000 7ff5161fe640 1 -- 172.21.15.148:0/2439032815 reap_dead start 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.686+0000 7ff517c61640 1 -- 172.21.15.148:0/2439032815 shutdown_connections 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.686+0000 7ff517c61640 1 -- 172.21.15.148:0/2439032815 >> 172.21.15.148:0/2439032815 conn(0x7ff5100d0780 msgr2=0x7ff5100d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.686+0000 7ff517c61640 1 -- 172.21.15.148:0/2439032815 shutdown_connections 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.686+0000 7ff517c61640 1 -- 172.21.15.148:0/2439032815 wait complete. 2024-03-04T12:12:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.687+0000 7ff517c61640 1 Processor -- start 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.687+0000 7ff517c61640 1 -- start start 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4fc0039b0 0x7ff5101573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5100c94f0 0x7ff5100cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51006acd0 con 0x7ff5100c9950 2024-03-04T12:12:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c75a0 con 0x7ff5100c94f0 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff517c61640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c3f30 con 0x7ff4fc0039b0 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40428/0 (socket says 172.21.15.148:40428) 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 -- 172.21.15.148:0/1097628773 learned_addr learned my addr 172.21.15.148:0/1097628773 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5151fc640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5100c94f0 0x7ff5100cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5159fd640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4fc0039b0 0x7ff5101573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 -- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4fc0039b0 msgr2=0x7ff5101573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4fc0039b0 0x7ff5101573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 -- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5100c94f0 msgr2=0x7ff5100cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5100c94f0 0x7ff5100cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 -- 172.21.15.148:0/1097628773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510157930 con 0x7ff5100c9950 2024-03-04T12:12:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5161fe640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff51015c0a0 tx=0x7ff50800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508019070 con 0x7ff5100c9950 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508003990 con 0x7ff5100c9950 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.688+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508003d80 con 0x7ff5100c9950 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.689+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100c4260 con 0x7ff5100c9950 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.689+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100c4820 con 0x7ff5100c9950 2024-03-04T12:12:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.689+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51006cd70 con 0x7ff5100c9950 2024-03-04T12:12:59.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.690+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff50801c070 con 0x7ff5100c9950 2024-03-04T12:12:59.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.690+0000 7ff5067fc640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 0x7ff4f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:12:59.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.690+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff50808a8d0 con 0x7ff5100c9950 2024-03-04T12:12:59.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.690+0000 7ff5159fd640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 0x7ff4f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:12:59.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.691+0000 7ff5159fd640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 0x7ff4f406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff50c034000 tx=0x7ff50c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:12:59.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.694+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff508010040 con 0x7ff5100c9950 2024-03-04T12:12:59.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:12:59.878+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff510074f60 con 0x7ff5100c9950 2024-03-04T12:13:00.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.502+0000 7ff5067fc640 1 -- 172.21.15.148:0/1097628773 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff508059300 con 0x7ff5100c9950 2024-03-04T12:13:00.505 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-03-04T12:13:00.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 msgr2=0x7ff4f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 0x7ff4f406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff50c034000 tx=0x7ff50c007000 comp rx=0 tx=0).stop 2024-03-04T12:13:00.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 msgr2=0x7ff5100cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff51015c0a0 tx=0x7ff50800f580 comp rx=0 tx=0).stop 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 shutdown_connections 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7ff4f406b500 0x7ff4f406d9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5100c9950 0x7ff5100cd7b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5100c94f0 0x7ff5100cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 --2- 172.21.15.148:0/1097628773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4fc0039b0 0x7ff5101573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 >> 172.21.15.148:0/1097628773 conn(0x7ff5100d0780 msgr2=0x7ff51015a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 shutdown_connections 2024-03-04T12:13:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.505+0000 7ff517c61640 1 -- 172.21.15.148:0/1097628773 wait complete. 2024-03-04T12:13:00.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-04T12:13:00.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.620+0000 7f14ccbe6640 1 Processor -- start 2024-03-04T12:13:00.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.620+0000 7f14ccbe6640 1 -- start start 2024-03-04T12:13:00.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14ccbe6640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f14c815d660 0x7f14c815e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14ccbe6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14c80aeb20 0x7f14c8071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14ccbe6640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c815da40 con 0x7f14c8071890 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14ccbe6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c8071ce0 con 0x7f14c80aeb20 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14ccbe6640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c815cbc0 con 0x7f14c815d660 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14c80aeb20 0x7f14c8071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14c80aeb20 0x7f14c8071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40430/0 (socket says 172.21.15.148:40430) 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 -- 172.21.15.148:0/3846070151 learned_addr learned my addr 172.21.15.148:0/3846070151 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14c659c640 1 --2- 172.21.15.148:0/3846070151 >> v2:172.21.15.191:3300/0 conn(0x7f14c815d660 0x7f14c815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 -- 172.21.15.148:0/3846070151 >> v2:172.21.15.191:3300/0 conn(0x7f14c815d660 msgr2=0x7f14c815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 --2- 172.21.15.148:0/3846070151 >> v2:172.21.15.191:3300/0 conn(0x7f14c815d660 0x7f14c815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 -- 172.21.15.148:0/3846070151 >> v1:172.21.15.191:6789/0 conn(0x7f14c8071890 legacy=0x7f14c8150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.621+0000 7f14bffff640 1 -- 172.21.15.148:0/3846070151 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c815ebb0 con 0x7f14c80aeb20 2024-03-04T12:13:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.622+0000 7f14bffff640 1 --2- 172.21.15.148:0/3846070151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c80aeb20 0x7f14c8071350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14c806d200 tx=0x7f14b002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2e369ca181d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:00.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.622+0000 7f14c5d9b640 1 -- 172.21.15.148:0/3846070151 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0038070 con 0x7f14c80aeb20 2024-03-04T12:13:00.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.622+0000 7f14c5d9b640 1 -- 172.21.15.148:0/3846070151 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0031520 con 0x7f14c80aeb20 2024-03-04T12:13:00.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.622+0000 7f14c5d9b640 1 -- 172.21.15.148:0/3846070151 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b00318f0 con 0x7f14c80aeb20 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 -- 172.21.15.148:0/3846070151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c80aeb20 msgr2=0x7f14c8071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/3846070151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c80aeb20 0x7f14c8071350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14c806d200 tx=0x7f14b002b7c0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 -- 172.21.15.148:0/3846070151 shutdown_connections 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/3846070151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c80aeb20 0x7f14c8071350 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/3846070151 >> v2:172.21.15.191:3300/0 conn(0x7f14c815d660 0x7f14c815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 -- 172.21.15.148:0/3846070151 >> 172.21.15.148:0/3846070151 conn(0x7f14c806acd0 msgr2=0x7f14c806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 -- 172.21.15.148:0/3846070151 shutdown_connections 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.623+0000 7f14ccbe6640 1 -- 172.21.15.148:0/3846070151 wait complete. 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.624+0000 7f14ccbe6640 1 Processor -- start 2024-03-04T12:13:00.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.624+0000 7f14ccbe6640 1 -- start start 2024-03-04T12:13:00.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14c8071890 0x7f14c81f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14c80aeb20 0x7f14c814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815acd0 con 0x7f14c815d660 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815a9f0 con 0x7f14c80aeb20 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c815afb0 con 0x7f14c8071890 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40442/0 (socket says 172.21.15.148:40442) 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 -- 172.21.15.148:0/2833382521 learned_addr learned my addr 172.21.15.148:0/2833382521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c659c640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14c8071890 0x7f14c81f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14bffff640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14c80aeb20 0x7f14c814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 -- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14c8071890 msgr2=0x7f14c81f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14c8071890 0x7f14c81f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 -- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14c80aeb20 msgr2=0x7f14c814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14c80aeb20 0x7f14c814f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 -- 172.21.15.148:0/2833382521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c81500f0 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14c6d9d640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c780 tx=0x7f14c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004b60 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0004d00 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c81503d0 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.625+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000aa90 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.626+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c8150990 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.627+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c8074f60 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.627+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f14c000ac30 con 0x7f14c815d660 2024-03-04T12:13:00.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.627+0000 7f14beffd640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 0x7f14a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.627+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14c008a190 con 0x7f14c815d660 2024-03-04T12:13:00.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.627+0000 7f14c659c640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 0x7f14a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:00.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.628+0000 7f14c659c640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 0x7f14a806d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14c80b4510 tx=0x7f14b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:00.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.632+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14c0020070 con 0x7f14c815d660 2024-03-04T12:13:00.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.875+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f14c8072460 con 0x7f14c815d660 2024-03-04T12:13:00.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.876+0000 7f14beffd640 1 -- 172.21.15.148:0/2833382521 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f14c0058b80 con 0x7f14c815d660 2024-03-04T12:13:00.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVlPTpZKBAAPdky8L7gzakTpgf1zEyLyw==", 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.879 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVlxXXRNBAA6K+J/CFuWZuA54WYAwv1OA==", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVl65bELxAATNugvAPFPxY3GrpmmdX9Lg==", 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.880 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBRuuVlrOGOABAACt3dfy2qT01wPlK7Kp+bPw==", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBUuuVlvsE+IxAAQjpgDYDw3OZLznNBrBlUyA==", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBXuuVliEcuIhAA9hXBDscdE49SjCXZUT9n6w==", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBauuVlMjJtIBAAnItsRXnawrqBExdXS3OHNQ==", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-03-04T12:13:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBduuVly8HZHhAA+QcJYwEKRsx7gI7WFiEwIQ==", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBjuuVlFwF1KBAAIReAK44KV03xz6DAEsgQ7Q==", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBmuuVlHTcRJxAAywci5A78GNGEDTDyVGAwBg==", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBpuuVlopHnJRAAi4cCZCxM9p1Z2x9jaEn9Fw==", 2024-03-04T12:13:00.885 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBsuuVlzF70IxAAxKllVg9IV4JxzNo8P8jHYw==", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBRuuVlHlfkBBAAcGWanX3ahyQy3t9Q4LBHWQ==", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBRuuVlrNcUDhAALPWdoKn42fRfmcZKUB6/iA==", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.887 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBRuuVlkW20ChAAg5QRFdWkrTWNys1Jd8i98g==", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBRuuVlBIrYExAAS6Y8zD2oL7igsrheXxgZoA==", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w==", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-04T12:13:00.889 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mds", 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVlMCmMHhAAlUtb3JyaJVorOJQ9IKzJbw==", 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mds" 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mgr", 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVlYTOMHhAAieAB5+MAmVSVu03CDSBxhA==", 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-04T12:13:00.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-osd", 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVlDzqMHhAAdJbwQJb50Oh1URldQ4yk/w==", 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-osd" 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd", 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVllECMHhAAV4n46EPUvuSP3KG/W7kLaQ==", 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVlI0eMHhAAEghqJUfFJnBF92pU2Ku/lA==", 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rgw", 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQByuuVlEE6MHhAA1e7paXBkC77APzLAT30WHg==", 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVl91znJBAAeC7+KjTaCJRMZzHjK2WifA==", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQBQuuVlxdJmIBAA2Dj8FYvTvXbue2805GEJQw==", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-03-04T12:13:00.893 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-04T12:13:00.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:00.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-03-04T12:13:00.894 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 msgr2=0x7f14a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 0x7f14a806d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14c80b4510 tx=0x7f14b8008040 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 msgr2=0x7f14c80af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c780 tx=0x7f14c000cc40 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 shutdown_connections 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f14a806b4d0 0x7f14a806d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c815d660 0x7f14c80af390 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14c80aeb20 0x7f14c814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 --2- 172.21.15.148:0/2833382521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14c8071890 0x7f14c81f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 >> 172.21.15.148:0/2833382521 conn(0x7f14c806acd0 msgr2=0x7f14c806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:00.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 shutdown_connections 2024-03-04T12:13:00.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.878+0000 7f14ccbe6640 1 -- 172.21.15.148:0/2833382521 wait complete. 2024-03-04T12:13:00.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-04T12:13:00.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.995+0000 7f0ced17d640 1 Processor -- start 2024-03-04T12:13:00.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.995+0000 7f0ced17d640 1 -- start start 2024-03-04T12:13:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ced17d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 0x7f0ce80cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:00.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ced17d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ce8070de0 0x7f0ce8071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ced17d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce815f020 con 0x7f0ce8154a80 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ced17d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce80cd010 con 0x7f0ce8070de0 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ced17d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce80cdaa0 con 0x7f0ce80cf650 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ce8070de0 0x7f0ce8071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ce8070de0 0x7f0ce8071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40450/0 (socket says 172.21.15.148:40450) 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 -- 172.21.15.148:0/1056710412 learned_addr learned my addr 172.21.15.148:0/1056710412 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0cdffff640 1 --2- 172.21.15.148:0/1056710412 >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 0x7f0ce80cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 -- 172.21.15.148:0/1056710412 >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 msgr2=0x7f0ce80cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 --2- 172.21.15.148:0/1056710412 >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 0x7f0ce80cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 -- 172.21.15.148:0/1056710412 >> v1:172.21.15.191:6789/0 conn(0x7f0ce8154a80 legacy=0x7f0ce815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0ce759e640 1 -- 172.21.15.148:0/1056710412 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80cdd80 con 0x7f0ce8070de0 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.996+0000 7f0cdffff640 1 --2- 172.21.15.148:0/1056710412 >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 0x7f0ce80cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ce759e640 1 --2- 172.21.15.148:0/1056710412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce8070de0 0x7f0ce8071270 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8067960 tx=0x7f0ce002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ca316f5a0cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ce659c640 1 -- 172.21.15.148:0/1056710412 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce002cb20 con 0x7f0ce8070de0 2024-03-04T12:13:01.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ce659c640 1 -- 172.21.15.148:0/1056710412 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce002ccc0 con 0x7f0ce8070de0 2024-03-04T12:13:01.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ce659c640 1 -- 172.21.15.148:0/1056710412 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0034610 con 0x7f0ce8070de0 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ced17d640 1 -- 172.21.15.148:0/1056710412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce8070de0 msgr2=0x7f0ce8071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.997+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1056710412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce8070de0 0x7f0ce8071270 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8067960 tx=0x7f0ce002bf80 comp rx=0 tx=0).stop 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 -- 172.21.15.148:0/1056710412 shutdown_connections 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1056710412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce8070de0 0x7f0ce8071270 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1056710412 >> v2:172.21.15.191:3300/0 conn(0x7f0ce80cf650 0x7f0ce80cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 -- 172.21.15.148:0/1056710412 >> 172.21.15.148:0/1056710412 conn(0x7f0ce806f5f0 msgr2=0x7f0ce806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 -- 172.21.15.148:0/1056710412 shutdown_connections 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.998+0000 7f0ced17d640 1 -- 172.21.15.148:0/1056710412 wait complete. 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.999+0000 7f0ced17d640 1 Processor -- start 2024-03-04T12:13:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:00.999+0000 7f0ced17d640 1 -- start start 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce8070de0 0x7f0ce8157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 0x7f0ce80d1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80ce620 con 0x7f0ce80cf650 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80ce340 con 0x7f0ce8070de0 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8154e60 con 0x7f0ce8154a80 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40454/0 (socket says 172.21.15.148:40454) 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 -- 172.21.15.148:0/1128842833 learned_addr learned my addr 172.21.15.148:0/1128842833 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ce759e640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 0x7f0ce80d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ce6d9d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce8070de0 0x7f0ce8157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 -- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 msgr2=0x7f0ce80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 0x7f0ce80d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 -- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce8070de0 msgr2=0x7f0ce8157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce8070de0 0x7f0ce8157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 -- 172.21.15.148:0/1128842833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80b4530 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ce759e640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 0x7f0ce80d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdffff640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e7e0 tx=0x7f0cd000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000f040 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0004030 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce8153f40 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0004400 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.000+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce80b4a80 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.002+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce8074f60 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.002+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0cd000c6d0 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.002+0000 7f0cdf7fe640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 0x7f0cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.002+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0cd0089de0 con 0x7f0ce80cf650 2024-03-04T12:13:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.002+0000 7f0ce6d9d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 0x7f0cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.003+0000 7f0ce6d9d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 0x7f0cc806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80016e0 tx=0x7f0cd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.006+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cd0017020 con 0x7f0ce80cf650 2024-03-04T12:13:01.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.241+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0ce8073b80 con 0x7f0ce80cf650 2024-03-04T12:13:01.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.243+0000 7f0cdf7fe640 1 -- 172.21.15.148:0/1128842833 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0cd0058740 con 0x7f0ce80cf650 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 msgr2=0x7f0cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 0x7f0cc806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80016e0 tx=0x7f0cd8009040 comp rx=0 tx=0).stop 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 msgr2=0x7f0ce8153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e7e0 tx=0x7f0cd000eca0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 shutdown_connections 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0cc806b4d0 0x7f0cc806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce8154a80 0x7f0ce80d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce80cf650 0x7f0ce8153a00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 --2- 172.21.15.148:0/1128842833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce8070de0 0x7f0ce8157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 >> 172.21.15.148:0/1128842833 conn(0x7f0ce806f5f0 msgr2=0x7f0ce80b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 shutdown_connections 2024-03-04T12:13:01.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.245+0000 7f0ced17d640 1 -- 172.21.15.148:0/1128842833 wait complete. 2024-03-04T12:13:01.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-04T12:13:01.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.357+0000 7f531ddfa640 1 Processor -- start 2024-03-04T12:13:01.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.357+0000 7f531ddfa640 1 -- start start 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f531ddfa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f531ddfa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53180cfab0 0x7f5318161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f531ddfa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53180cdaa0 con 0x7f53180cf650 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f531ddfa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5318071df0 con 0x7f53180cfab0 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f531ddfa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180cd010 con 0x7f5318154a80 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5317fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53180cfab0 0x7f5318161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5316ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f53180cf650 0x7f53181616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59466/0 (socket says 172.21.15.148:59466) 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5316ffd640 1 -- 172.21.15.148:0/1553673341 learned_addr learned my addr 172.21.15.148:0/1553673341 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:01.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f53177fe640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5317fff640 1 -- 172.21.15.148:0/1553673341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 msgr2=0x7f531815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5317fff640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.358+0000 7f5317fff640 1 -- 172.21.15.148:0/1553673341 >> v1:172.21.15.148:6789/0 conn(0x7f53180cf650 legacy=0x7f53181616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.359+0000 7f5317fff640 1 -- 172.21.15.148:0/1553673341 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180cdd80 con 0x7f53180cfab0 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.359+0000 7f5317fff640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f5318161dd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5310009d90 tx=0x7f531002bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6965ac337e83d5be server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.359+0000 7f53167fc640 1 -- 172.21.15.148:0/1553673341 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310039070 con 0x7f53180cfab0 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.359+0000 7f53167fc640 1 -- 172.21.15.148:0/1553673341 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531002ca10 con 0x7f53180cfab0 2024-03-04T12:13:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.359+0000 7f53167fc640 1 -- 172.21.15.148:0/1553673341 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531002cde0 con 0x7f53180cfab0 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 -- 172.21.15.148:0/1553673341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 msgr2=0x7f5318161dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f5318161dd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5310009d90 tx=0x7f531002bce0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 -- 172.21.15.148:0/1553673341 shutdown_connections 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f5318161dd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 --2- 172.21.15.148:0/1553673341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 -- 172.21.15.148:0/1553673341 >> 172.21.15.148:0/1553673341 conn(0x7f531806f5f0 msgr2=0x7f531806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 -- 172.21.15.148:0/1553673341 shutdown_connections 2024-03-04T12:13:01.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.360+0000 7f531ddfa640 1 -- 172.21.15.148:0/1553673341 wait complete. 2024-03-04T12:13:01.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.361+0000 7f531ddfa640 1 Processor -- start 2024-03-04T12:13:01.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.361+0000 7f531ddfa640 1 -- start start 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 0x7f5318153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180ce620 con 0x7f53180cfab0 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180ce340 con 0x7f53180cf650 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f531ddfa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180cff40 con 0x7f5318154a80 2024-03-04T12:13:01.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40478/0 (socket says 172.21.15.148:40478) 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 -- 172.21.15.148:0/167630205 learned_addr learned my addr 172.21.15.148:0/167630205 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 -- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 msgr2=0x7f531815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f53177fe640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 0x7f5318153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5317fff640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 -- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 msgr2=0x7f5318153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 0x7f5318153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5316ffd640 1 -- 172.21.15.148:0/167630205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53181f72f0 con 0x7f53180cfab0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f5317fff640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.362+0000 7f53177fe640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 0x7f5318153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f5316ffd640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c780 tx=0x7f530c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c018070 con 0x7f53180cfab0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c0072d0 con 0x7f53180cfab0 2024-03-04T12:13:01.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c0047e0 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53181f7590 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.363+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53181f7b50 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.364+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531806b8e0 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.364+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f530c004980 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.364+0000 7f52fffff640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.365+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f530c089550 con 0x7f53180cfab0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.365+0000 7f53177fe640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.365+0000 7f53177fe640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 0x7f530406d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f530800b820 tx=0x7f5308008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.369+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f531806b8e0 con 0x7f53180cfab0 2024-03-04T12:13:01.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.606+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5318074f60 con 0x7f53180cfab0 2024-03-04T12:13:01.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.609+0000 7f52fffff640 1 -- 172.21.15.148:0/167630205 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f530c004c70 con 0x7f53180cfab0 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 msgr2=0x7f530406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 0x7f530406d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f530800b820 tx=0x7f5308008040 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 msgr2=0x7f531815a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c780 tx=0x7f530c00cc40 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 shutdown_connections 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318154a80 0x7f531815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53180cfab0 0x7f531815a890 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 --2- 172.21.15.148:0/167630205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53180cf650 0x7f5318153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.610+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 >> 172.21.15.148:0/167630205 conn(0x7f531806f5f0 msgr2=0x7f531806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.611+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 shutdown_connections 2024-03-04T12:13:01.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.611+0000 7f531ddfa640 1 -- 172.21.15.148:0/167630205 wait complete. 2024-03-04T12:13:01.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-04T12:13:01.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.722+0000 7f8c254f8640 1 Processor -- start 2024-03-04T12:13:01.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.722+0000 7f8c254f8640 1 -- start start 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c254f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c2015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c254f8640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8c200cf650 0x7f8c200cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c254f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c200cdaa0 con 0x7f8c201616c0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c254f8640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200cfff0 con 0x7f8c200cf650 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c254f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20071df0 con 0x7f8c20154a80 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c1f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8c201616c0 0x7f8c20161b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59472/0 (socket says 172.21.15.148:59472) 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c1f7fe640 1 -- 172.21.15.148:0/2901947185 learned_addr learned my addr 172.21.15.148:0/2901947185 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c1e7fc640 1 --2- 172.21.15.148:0/2901947185 >> v2:172.21.15.191:3300/0 conn(0x7f8c200cf650 0x7f8c200cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c1effd640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c2015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.723+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109269281 0 0) 0x7f8c200cdaa0 con 0x7f8c201616c0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c08003520 con 0x7f8c201616c0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310871765 0 0) 0x7f8c08003520 con 0x7f8c201616c0 2024-03-04T12:13:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 msgr2=0x7f8c2015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c2015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 >> v2:172.21.15.191:3300/0 conn(0x7f8c200cf650 msgr2=0x7f8c200cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 --2- 172.21.15.148:0/2901947185 >> v2:172.21.15.191:3300/0 conn(0x7f8c200cf650 0x7f8c200cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200cd010 con 0x7f8c201616c0 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f8c18001670 con 0x7f8c201616c0 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 >> v1:172.21.15.148:6789/0 conn(0x7f8c201616c0 legacy=0x7f8c20161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c08003f60 0x7f8c08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20071df0 con 0x7f8c08004d40 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0800b9c0 con 0x7f8c08003f60 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0800bb40 con 0x7f8c20154a80 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1e7fc640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c08003f60 0x7f8c08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1effd640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 -- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 msgr2=0x7f8c080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c20154a80 0x7f8c080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 -- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c08003f60 msgr2=0x7f8c08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c08003f60 0x7f8c08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.724+0000 7f8c1f7fe640 1 -- 172.21.15.148:0/2901947185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08003520 con 0x7f8c08004d40 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.725+0000 7f8c1f7fe640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800c240 tx=0x7f8c08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2955f40af28952 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.725+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18007cf0 con 0x7f8c08004d40 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.725+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18030070 con 0x7f8c08004d40 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.725+0000 7f8c1dffb640 1 -- 172.21.15.148:0/2901947185 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18005d30 con 0x7f8c08004d40 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 -- 172.21.15.148:0/2901947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c08004d40 msgr2=0x7f8c080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 --2- 172.21.15.148:0/2901947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800c240 tx=0x7f8c08005830 comp rx=0 tx=0).stop 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c1f7fe640 1 -- 172.21.15.148:0/2901947185 reap_dead start 2024-03-04T12:13:01.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 -- 172.21.15.148:0/2901947185 shutdown_connections 2024-03-04T12:13:01.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 -- 172.21.15.148:0/2901947185 >> 172.21.15.148:0/2901947185 conn(0x7f8c2006f5f0 msgr2=0x7f8c2006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 -- 172.21.15.148:0/2901947185 shutdown_connections 2024-03-04T12:13:01.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.726+0000 7f8c254f8640 1 -- 172.21.15.148:0/2901947185 wait complete. 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.727+0000 7f8c254f8640 1 Processor -- start 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.727+0000 7f8c254f8640 1 -- start start 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c201e7b30 0x7f8c200c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200c58a0 0x7f8c20146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200ce340 con 0x7f8c201e7750 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200ce060 con 0x7f8c200c58a0 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200ce620 con 0x7f8c201e7b30 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40508/0 (socket says 172.21.15.148:40508) 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 -- 172.21.15.148:0/3634558969 learned_addr learned my addr 172.21.15.148:0/3634558969 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1e7fc640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c201e7b30 0x7f8c200c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1f7fe640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200c58a0 0x7f8c20146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 -- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c201e7b30 msgr2=0x7f8c200c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c201e7b30 0x7f8c200c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 -- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200c58a0 msgr2=0x7f8c20146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200c58a0 0x7f8c20146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 -- 172.21.15.148:0/3634558969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200c5f40 con 0x7f8c201e7750 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c1effd640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8c10011e60 tx=0x7f8c1000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10015630 con 0x7f8c201e7750 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100157d0 con 0x7f8c201e7750 2024-03-04T12:13:01.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c200c6220 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10015ba0 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.728+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c200c67e0 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.729+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20074f60 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.730+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8c1001d400 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.730+0000 7f8c077fe640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.730+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8c1008b210 con 0x7f8c201e7750 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.730+0000 7f8c1e7fc640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.731+0000 7f8c1e7fc640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c20154490 tx=0x7f8c14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:01.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.735+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c10019030 con 0x7f8c201e7750 2024-03-04T12:13:01.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.971+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8c200c5c80 con 0x7f8c201e7750 2024-03-04T12:13:01.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.973+0000 7f8c077fe640 1 -- 172.21.15.148:0/3634558969 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8c10010040 con 0x7f8c201e7750 2024-03-04T12:13:01.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.975+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 msgr2=0x7f8c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c20154490 tx=0x7f8c14008040 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 msgr2=0x7f8c200b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8c10011e60 tx=0x7f8c1000f580 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 shutdown_connections 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200c58a0 0x7f8c20146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c201e7b30 0x7f8c200c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 --2- 172.21.15.148:0/3634558969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c201e7750 0x7f8c200b4ba0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 >> 172.21.15.148:0/3634558969 conn(0x7f8c2006f5f0 msgr2=0x7f8c200b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 shutdown_connections 2024-03-04T12:13:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:01.976+0000 7f8c254f8640 1 -- 172.21.15.148:0/3634558969 wait complete. 2024-03-04T12:13:01.989 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-04T12:13:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.087+0000 7f80911f3640 1 Processor -- start 2024-03-04T12:13:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.087+0000 7f80911f3640 1 -- start start 2024-03-04T12:13:02.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f80911f3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f808c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f80911f3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c06d860 con 0x7f808c15b560 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f80911f3640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c15ed90 con 0x7f808c0b3c60 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f80911f3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b40d0 con 0x7f808c15b940 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f808ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f808c15b560 0x7f808c15e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59484/0 (socket says 172.21.15.148:59484) 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f808ad9d640 1 -- 172.21.15.148:0/708909472 learned_addr learned my addr 172.21.15.148:0/708909472 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f808a59c640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f808c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401494686 0 0) 0x7f808c06d860 con 0x7f808c15b560 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8074003520 con 0x7f808c15b560 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.088+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378855862 0 0) 0x7f808c15ed90 con 0x7f808c0b3c60 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f808c06d860 con 0x7f808c0b3c60 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330712018 0 0) 0x7f8074003520 con 0x7f808c15b560 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 msgr2=0x7f808c0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f808c0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 >> v1:172.21.15.191:6789/0 conn(0x7f808c0b3c60 legacy=0x7f808c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c15f070 con 0x7f808c15b560 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f807c0019f0 con 0x7f808c15b560 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 >> v1:172.21.15.148:6789/0 conn(0x7f808c15b560 legacy=0x7f808c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f8074003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f8074004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8074004d40 0x7f80740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b40d0 con 0x7f8074003b70 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807400ba70 con 0x7f8074004d40 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807400bc70 con 0x7f808c15b940 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f8074003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808b59e640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8074004d40 0x7f80740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808a59c640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f8074004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 -- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 msgr2=0x7f8074004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c15b940 0x7f8074004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 -- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8074004d40 msgr2=0x7f80740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8074004d40 0x7f80740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.089+0000 7f808ad9d640 1 -- 172.21.15.148:0/708909472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074003520 con 0x7f8074003b70 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f808ad9d640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f8074003f70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a7c0 tx=0x7f8074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b318cbd576fee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00a410 con 0x7f8074003b70 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c034650 con 0x7f8074003b70 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f8089d9b640 1 -- 172.21.15.148:0/708909472 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c02ce60 con 0x7f8074003b70 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f80911f3640 1 -- 172.21.15.148:0/708909472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 msgr2=0x7f8074003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f80911f3640 1 --2- 172.21.15.148:0/708909472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f8074003f70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a7c0 tx=0x7f8074005830 comp rx=0 tx=0).stop 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f808b59e640 1 -- 172.21.15.148:0/708909472 reap_dead start 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f80911f3640 1 -- 172.21.15.148:0/708909472 shutdown_connections 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.090+0000 7f80911f3640 1 -- 172.21.15.148:0/708909472 >> 172.21.15.148:0/708909472 conn(0x7f808c0cfb40 msgr2=0x7f808c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.091+0000 7f80911f3640 1 -- 172.21.15.148:0/708909472 shutdown_connections 2024-03-04T12:13:02.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.091+0000 7f80911f3640 1 -- 172.21.15.148:0/708909472 wait complete. 2024-03-04T12:13:02.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.092+0000 7f80911f3640 1 Processor -- start 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.092+0000 7f80911f3640 1 -- start start 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f808c0c4f60 0x7f808c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c149260 0x7f808c1496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c071a90 con 0x7f8074003b70 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c06de00 con 0x7f808c0c4f60 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80911f3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c15e970 con 0x7f808c149260 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40526/0 (socket says 172.21.15.148:40526) 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 -- 172.21.15.148:0/423533148 learned_addr learned my addr 172.21.15.148:0/423533148 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808a59c640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f808c0c4f60 0x7f808c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808b59e640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c149260 0x7f808c1496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 -- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c149260 msgr2=0x7f808c1496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c149260 0x7f808c1496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 -- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f808c0c4f60 msgr2=0x7f808c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f808c0c4f60 0x7f808c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 -- 172.21.15.148:0/423533148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c15f920 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f808ad9d640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a7c0 tx=0x7f808c15f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.093+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c034a00 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.094+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c03a040 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.094+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c15fc00 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.094+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c034da0 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.094+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c1601c0 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.094+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c074f60 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.095+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f807c03f420 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.095+0000 7f80737fe640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 0x7f807806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.095+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f807c0ac690 con 0x7f8074003b70 2024-03-04T12:13:02.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.095+0000 7f808a59c640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 0x7f807806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.096+0000 7f808a59c640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 0x7f807806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f808c14d300 tx=0x7f8080009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.100+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f807c07b760 con 0x7f8074003b70 2024-03-04T12:13:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.334+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f808c0cf920 con 0x7f8074003b70 2024-03-04T12:13:02.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.336+0000 7f80737fe640 1 -- 172.21.15.148:0/423533148 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f807c07b060 con 0x7f8074003b70 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 msgr2=0x7f807806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 0x7f807806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f808c14d300 tx=0x7f8080009190 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 msgr2=0x7f808c15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f807c00a7c0 tx=0x7f808c15f240 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 shutdown_connections 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f807806b4d0 0x7f807806d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f808c149260 0x7f808c1496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f808c0c4f60 0x7f808c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 --2- 172.21.15.148:0/423533148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8074003b70 0x7f808c15f2b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 >> 172.21.15.148:0/423533148 conn(0x7f808c0cfb40 msgr2=0x7f808c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 shutdown_connections 2024-03-04T12:13:02.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.338+0000 7f80911f3640 1 -- 172.21.15.148:0/423533148 wait complete. 2024-03-04T12:13:02.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-04T12:13:02.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.450+0000 7fd7dafb9640 1 Processor -- start 2024-03-04T12:13:02.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.450+0000 7fd7dafb9640 1 -- start start 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7dafb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d406d3b0 0x7fd7d40ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7dafb9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7d40cc500 0x7fd7d40cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7dafb9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d40d0140 con 0x7fd7d40cfd60 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7dafb9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d40cf440 con 0x7fd7d40cc500 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7dafb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d415cbb0 con 0x7fd7d406d3b0 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7d9556640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7d40cc500 0x7fd7d40cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7d9556640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7d40cc500 0x7fd7d40cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53340/0 (socket says 172.21.15.148:53340) 2024-03-04T12:13:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7d8d55640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd7d40cfd60 0x7fd7d40cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52138/0 (socket says 172.21.15.148:52138) 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7d9556640 1 -- 172.21.15.148:0/30955938 learned_addr learned my addr 172.21.15.148:0/30955938 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.451+0000 7fd7d3fff640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d406d3b0 0x7fd7d40ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d9556640 1 -- 172.21.15.148:0/30955938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d406d3b0 msgr2=0x7fd7d40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d9556640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d406d3b0 0x7fd7d40ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d9556640 1 -- 172.21.15.148:0/30955938 >> v1:172.21.15.148:6789/0 conn(0x7fd7d40cfd60 legacy=0x7fd7d40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d37fe640 1 -- 172.21.15.148:0/30955938 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245086189 0 0) 0x7fd7d40d0140 con 0x7fd7d40cfd60 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d9556640 1 -- 172.21.15.148:0/30955938 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d415c280 con 0x7fd7d40cc500 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d9556640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40cef00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009e50 tx=0x7fd7cc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f253ee147f1b1a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d37fe640 1 -- 172.21.15.148:0/30955938 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc02cb00 con 0x7fd7d40cc500 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d37fe640 1 -- 172.21.15.148:0/30955938 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc02cca0 con 0x7fd7d40cc500 2024-03-04T12:13:02.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.452+0000 7fd7d37fe640 1 -- 172.21.15.148:0/30955938 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc034610 con 0x7fd7d40cc500 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 -- 172.21.15.148:0/30955938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 msgr2=0x7fd7d40cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40cef00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009e50 tx=0x7fd7cc02bf60 comp rx=0 tx=0).stop 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 -- 172.21.15.148:0/30955938 shutdown_connections 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40cef00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/30955938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d406d3b0 0x7fd7d40ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 -- 172.21.15.148:0/30955938 >> 172.21.15.148:0/30955938 conn(0x7fd7d406acd0 msgr2=0x7fd7d406b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 -- 172.21.15.148:0/30955938 shutdown_connections 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.453+0000 7fd7dafb9640 1 -- 172.21.15.148:0/30955938 wait complete. 2024-03-04T12:13:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.454+0000 7fd7dafb9640 1 Processor -- start 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.454+0000 7fd7dafb9640 1 -- start start 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7d406d3b0 0x7fd7d414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d40cfd60 0x7fd7d4140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d415bf30 con 0x7fd7d40cc500 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d415c840 con 0x7fd7d406d3b0 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7dafb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d415f520 con 0x7fd7d40cfd60 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53352/0 (socket says 172.21.15.148:53352) 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 -- 172.21.15.148:0/3615224912 learned_addr learned my addr 172.21.15.148:0/3615224912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d9556640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d40cfd60 0x7fd7d4140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d8d55640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7d406d3b0 0x7fd7d414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 -- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d40cfd60 msgr2=0x7fd7d4140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d40cfd60 0x7fd7d4140eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 -- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7d406d3b0 msgr2=0x7fd7d414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7d406d3b0 0x7fd7d414dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.455+0000 7fd7d3fff640 1 -- 172.21.15.148:0/3615224912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d40b16b0 con 0x7fd7d40cc500 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7d3fff640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800d800 tx=0x7fd7c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800b6c0 con 0x7fd7d40cc500 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c8019070 con 0x7fd7d40cc500 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800baa0 con 0x7fd7d40cc500 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d40b1990 con 0x7fd7d40cc500 2024-03-04T12:13:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.456+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d414e610 con 0x7fd7d40cc500 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.457+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d4074f60 con 0x7fd7d40cc500 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.457+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd7c800bc40 con 0x7fd7d40cc500 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.457+0000 7fd7d17fa640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 0x7fd7c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.458+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd7c808a4f0 con 0x7fd7d40cc500 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.458+0000 7fd7d8d55640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 0x7fd7c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.458+0000 7fd7d8d55640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 0x7fd7c006d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000f60 tx=0x7fd7c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.462+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7c8059530 con 0x7fd7d40cc500 2024-03-04T12:13:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.702+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd7d40cc2e0 con 0x7fd7d40cc500 2024-03-04T12:13:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.704+0000 7fd7d17fa640 1 -- 172.21.15.148:0/3615224912 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd7c8058e30 con 0x7fd7d40cc500 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 msgr2=0x7fd7c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 0x7fd7c006d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000f60 tx=0x7fd7c4009040 comp rx=0 tx=0).stop 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 msgr2=0x7fd7d40b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800d800 tx=0x7fd7c800dcc0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 shutdown_connections 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd7c006b500 0x7fd7c006d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7d40cfd60 0x7fd7d4140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d40cc500 0x7fd7d40b0c60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 --2- 172.21.15.148:0/3615224912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7d406d3b0 0x7fd7d414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 >> 172.21.15.148:0/3615224912 conn(0x7fd7d406acd0 msgr2=0x7fd7d4159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 shutdown_connections 2024-03-04T12:13:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.707+0000 7fd7dafb9640 1 -- 172.21.15.148:0/3615224912 wait complete. 2024-03-04T12:13:02.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-04T12:13:02.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 Processor -- start 2024-03-04T12:13:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 -- start start 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f82680cc500 0x7f8268166680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680d0140 con 0x7f826806d3b0 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826815c280 con 0x7f82680cfd60 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.820+0000 7f826fb09640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826815cbb0 con 0x7f82680cc500 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826d0a4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f826806d3b0 0x7f82680ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52144/0 (socket says 172.21.15.148:52144) 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826d0a4640 1 -- 172.21.15.148:0/4174986571 learned_addr learned my addr 172.21.15.148:0/4174986571 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826e0a6640 1 --2- 172.21.15.148:0/4174986571 >> v2:172.21.15.191:3300/0 conn(0x7f82680cc500 0x7f8268166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362119923 0 0) 0x7f82680d0140 con 0x7f826806d3b0 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8254003520 con 0x7f826806d3b0 2024-03-04T12:13:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892139808 0 0) 0x7f8254003520 con 0x7f826806d3b0 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 >> v2:172.21.15.191:3300/0 conn(0x7f82680cc500 msgr2=0x7f8268166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 --2- 172.21.15.148:0/4174986571 >> v2:172.21.15.191:3300/0 conn(0x7f82680cc500 0x7f8268166680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 >> v1:172.21.15.191:6789/0 conn(0x7f82680cfd60 legacy=0x7f82680cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826815c560 con 0x7f826806d3b0 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f8264001670 con 0x7f826806d3b0 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826e0a6640 1 --2- 172.21.15.148:0/4174986571 >> v2:172.21.15.191:3300/0 conn(0x7f82680cc500 0x7f8268166680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 >> v1:172.21.15.148:6789/0 conn(0x7f826806d3b0 legacy=0x7f82680ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8254003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8254004800 0x7f8254004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82540051c0 0x7f825400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815cbb0 con 0x7f82540039b0 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825400c060 con 0x7f82540051c0 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825400c260 con 0x7f8254004800 2024-03-04T12:13:02.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826d8a5640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8254003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826e0a6640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82540051c0 0x7f825400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.821+0000 7f826d0a4640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8254004800 0x7f8254004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 -- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8254004800 msgr2=0x7f8254004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8254004800 0x7f8254004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 -- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82540051c0 msgr2=0x7f825400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82540051c0 0x7f825400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826e0a6640 1 -- 172.21.15.148:0/4174986571 reap_dead start 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 -- 172.21.15.148:0/4174986571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254003520 con 0x7f82540039b0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826d8a5640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8254003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8258001750 tx=0x7f8254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c81a5fbbb202 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.822+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258001370 con 0x7f82540039b0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580052d0 con 0x7f82540039b0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826c8a3640 1 -- 172.21.15.148:0/4174986571 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258003830 con 0x7f82540039b0 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 -- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 msgr2=0x7f8254003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8254003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8258001750 tx=0x7f8254005830 comp rx=0 tx=0).stop 2024-03-04T12:13:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 -- 172.21.15.148:0/4174986571 shutdown_connections 2024-03-04T12:13:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 --2- 172.21.15.148:0/4174986571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8254003dd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 -- 172.21.15.148:0/4174986571 >> 172.21.15.148:0/4174986571 conn(0x7f826806acd0 msgr2=0x7f826806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 -- 172.21.15.148:0/4174986571 shutdown_connections 2024-03-04T12:13:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.823+0000 7f826fb09640 1 -- 172.21.15.148:0/4174986571 wait complete. 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.824+0000 7f826fb09640 1 Processor -- start 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.824+0000 7f826fb09640 1 -- start start 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 0x7f82680ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f826815afe0 0x7f826815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cbdf0 con 0x7f82540039b0 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815bf30 con 0x7f82680ae910 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826fb09640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cbf70 con 0x7f826815afe0 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53370/0 (socket says 172.21.15.148:53370) 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 -- 172.21.15.148:0/108892054 learned_addr learned my addr 172.21.15.148:0/108892054 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d0a4640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 0x7f82680ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826e0a6640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f826815afe0 0x7f826815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 -- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f826815afe0 msgr2=0x7f826815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f826815afe0 0x7f826815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 -- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 msgr2=0x7f82680ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 0x7f82680ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d8a5640 1 -- 172.21.15.148:0/108892054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82681576e0 con 0x7f82540039b0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.825+0000 7f826d0a4640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 0x7f82680ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f826d8a5640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258041020 con 0x7f82540039b0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258005480 con 0x7f82540039b0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258007a30 con 0x7f82540039b0 2024-03-04T12:13:02.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82681579c0 con 0x7f82540039b0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.826+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8268157f80 con 0x7f82540039b0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.827+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8268074f60 con 0x7f82540039b0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.827+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8258044020 con 0x7f82540039b0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.828+0000 7f825e7fc640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.828+0000 7f826d0a4640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.828+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f82580abac0 con 0x7f82540039b0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.828+0000 7f826d0a4640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 0x7f824c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f826815a0e0 tx=0x7f8264009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:02.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:02.832+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f825804b030 con 0x7f82540039b0 2024-03-04T12:13:03.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.068+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f82680ce640 con 0x7f82540039b0 2024-03-04T12:13:03.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.070+0000 7f825e7fc640 1 -- 172.21.15.148:0/108892054 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f825807aa70 con 0x7f82540039b0 2024-03-04T12:13:03.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 msgr2=0x7f824c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 0x7f824c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f826815a0e0 tx=0x7f8264009040 comp rx=0 tx=0).stop 2024-03-04T12:13:03.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 msgr2=0x7f8268142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258001120 comp rx=0 tx=0).stop 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 shutdown_connections 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f826815afe0 0x7f826815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82680ae910 0x7f82680ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 --2- 172.21.15.148:0/108892054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82540039b0 0x7f8268142af0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 >> 172.21.15.148:0/108892054 conn(0x7f826806acd0 msgr2=0x7f826815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 shutdown_connections 2024-03-04T12:13:03.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.072+0000 7f826fb09640 1 -- 172.21.15.148:0/108892054 wait complete. 2024-03-04T12:13:03.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:13:03.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.183+0000 7f57a79d4640 1 Processor -- start 2024-03-04T12:13:03.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 -- start start 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57a00cc690 0x7f57a00d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a00cfb40 con 0x7f57a00d0e40 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00d2880 con 0x7f57a00cc690 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a79d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cfe20 con 0x7f57a015bbb0 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a5f71640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57a00cc690 0x7f57a00d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a5f71640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57a00cc690 0x7f57a00d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53376/0 (socket says 172.21.15.148:53376) 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.184+0000 7f57a5f71640 1 -- 172.21.15.148:0/2389904120 learned_addr learned my addr 172.21.15.148:0/2389904120 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.185+0000 7f57a5770640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.185+0000 7f57a5f71640 1 -- 172.21.15.148:0/2389904120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 msgr2=0x7f57a00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.185+0000 7f57a5f71640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.185+0000 7f57a5f71640 1 -- 172.21.15.148:0/2389904120 >> v1:172.21.15.191:6789/0 conn(0x7f57a00d0e40 legacy=0x7f57a00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.185+0000 7f57a5f71640 1 -- 172.21.15.148:0/2389904120 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00d0100 con 0x7f57a00cc690 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a5f71640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00d1ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41403642b88e2fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f5797fff640 1 -- 172.21.15.148:0/2389904120 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798035020 con 0x7f57a00cc690 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f5797fff640 1 -- 172.21.15.148:0/2389904120 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798031640 con 0x7f57a00cc690 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f5797fff640 1 -- 172.21.15.148:0/2389904120 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579802d040 con 0x7f57a00cc690 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 -- 172.21.15.148:0/2389904120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 msgr2=0x7f57a00d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00d1ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802b8f0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 -- 172.21.15.148:0/2389904120 shutdown_connections 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00d1ca0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2389904120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a00d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 -- 172.21.15.148:0/2389904120 >> 172.21.15.148:0/2389904120 conn(0x7f57a00b3480 msgr2=0x7f57a00b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 -- 172.21.15.148:0/2389904120 shutdown_connections 2024-03-04T12:13:03.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.186+0000 7f57a79d4640 1 -- 172.21.15.148:0/2389904120 wait complete. 2024-03-04T12:13:03.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.187+0000 7f57a79d4640 1 Processor -- start 2024-03-04T12:13:03.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 -- start start 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57a00d0e40 0x7f57a00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a0158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015bf90 con 0x7f57a00cc690 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00d21e0 con 0x7f57a00d0e40 2024-03-04T12:13:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a79d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0155d10 con 0x7f57a015bbb0 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a5770640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a5770640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53384/0 (socket says 172.21.15.148:53384) 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a5770640 1 -- 172.21.15.148:0/2210539886 learned_addr learned my addr 172.21.15.148:0/2210539886 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a4f6f640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57a00d0e40 0x7f57a00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.188+0000 7f57a5f71640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a0158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 -- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 msgr2=0x7f57a0158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a0158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 -- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57a00d0e40 msgr2=0x7f57a00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57a00d0e40 0x7f57a00cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 -- 172.21.15.148:0/2210539886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00ca090 con 0x7f57a00cc690 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5f71640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a0158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a5770640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f579c007e40 tx=0x7f579c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c014620 con 0x7f57a00cc690 2024-03-04T12:13:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c019070 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00ca370 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0149e0 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.189+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00cac10 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.190+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0074f60 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.190+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f579c014b80 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.191+0000 7f5795ffb640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 0x7f578406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.191+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f579c08a960 con 0x7f57a00cc690 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.191+0000 7f57a4f6f640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 0x7f578406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.191+0000 7f57a4f6f640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 0x7f578406d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b6e40 tx=0x7f5790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.196+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f579c059910 con 0x7f57a00cc690 2024-03-04T12:13:03.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.367+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a00ca540 con 0x7f57a00cc690 2024-03-04T12:13:03.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.368+0000 7f5795ffb640 1 -- 172.21.15.148:0/2210539886 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13224 (secure 0 0 0) 0x7f579c059210 con 0x7f57a00cc690 2024-03-04T12:13:03.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:03.371 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":34,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:13:00.511286+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 msgr2=0x7f578406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 0x7f578406d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b6e40 tx=0x7f5790009040 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 msgr2=0x7f57a00b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f579c007e40 tx=0x7f579c00e4b0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 shutdown_connections 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f578406b4d0 0x7f578406d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57a015bbb0 0x7f57a0158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57a00d0e40 0x7f57a00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 --2- 172.21.15.148:0/2210539886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a00cc690 0x7f57a00b1ed0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 >> 172.21.15.148:0/2210539886 conn(0x7f57a00b3480 msgr2=0x7f57a00b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 shutdown_connections 2024-03-04T12:13:03.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.370+0000 7f57a79d4640 1 -- 172.21.15.148:0/2210539886 wait complete. 2024-03-04T12:13:03.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T12:13:03.384 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T12:13:03.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.484+0000 7f2e2bf76640 1 Processor -- start 2024-03-04T12:13:03.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.484+0000 7f2e2bf76640 1 -- start start 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2bf76640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e240b0150 0x7f2e240cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2bf76640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2415cf50 con 0x7f2e2415c7f0 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2bf76640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e240cc390 con 0x7f2e2406b3e0 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2bf76640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240cc670 con 0x7f2e240b0150 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2a513640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e240b0150 0x7f2e240cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e29d12640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2e2415c7f0 0x7f2e2406acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52152/0 (socket says 172.21.15.148:52152) 2024-03-04T12:13:03.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e29d12640 1 -- 172.21.15.148:0/1003941650 learned_addr learned my addr 172.21.15.148:0/1003941650 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:03.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2a513640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e240b0150 0x7f2e240cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53398/0 (socket says 172.21.15.148:53398) 2024-03-04T12:13:03.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2a513640 1 -- 172.21.15.148:0/1003941650 >> v1:172.21.15.191:6789/0 conn(0x7f2e2406b3e0 legacy=0x7f2e240cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2a513640 1 -- 172.21.15.148:0/1003941650 >> v1:172.21.15.148:6789/0 conn(0x7f2e2415c7f0 legacy=0x7f2e2406acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e2a513640 1 -- 172.21.15.148:0/1003941650 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240cc950 con 0x7f2e240b0150 2024-03-04T12:13:03.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.485+0000 7f2e28d10640 1 -- 172.21.15.148:0/1003941650 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290490435 0 0) 0x7f2e2415cf50 con 0x7f2e2415c7f0 2024-03-04T12:13:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.486+0000 7f2e2a513640 1 --2- 172.21.15.148:0/1003941650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e240b0150 0x7f2e240cd860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009d90 tx=0x7f2e1c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a8bdcdb3eacfeb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.486+0000 7f2e28d10640 1 -- 172.21.15.148:0/1003941650 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c039070 con 0x7f2e240b0150 2024-03-04T12:13:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.486+0000 7f2e28d10640 1 -- 172.21.15.148:0/1003941650 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c02cb40 con 0x7f2e240b0150 2024-03-04T12:13:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.486+0000 7f2e28d10640 1 -- 172.21.15.148:0/1003941650 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c034440 con 0x7f2e240b0150 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 -- 172.21.15.148:0/1003941650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e240b0150 msgr2=0x7f2e240cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/1003941650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e240b0150 0x7f2e240cd860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009d90 tx=0x7f2e1c02bd90 comp rx=0 tx=0).stop 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 -- 172.21.15.148:0/1003941650 shutdown_connections 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/1003941650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e240b0150 0x7f2e240cd860 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 -- 172.21.15.148:0/1003941650 >> 172.21.15.148:0/1003941650 conn(0x7f2e240b22e0 msgr2=0x7f2e240b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 -- 172.21.15.148:0/1003941650 shutdown_connections 2024-03-04T12:13:03.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.487+0000 7f2e2bf76640 1 -- 172.21.15.148:0/1003941650 wait complete. 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.488+0000 7f2e2bf76640 1 Processor -- start 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.488+0000 7f2e2bf76640 1 -- start start 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 0x7f2e240adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 0x7f2e240b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240cdda0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2415cbe0 con 0x7f2e240ae030 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2bf76640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240cdf20 con 0x7f2e240b0150 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53414/0 (socket says 172.21.15.148:53414) 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 -- 172.21.15.148:0/2341603487 learned_addr learned my addr 172.21.15.148:0/2341603487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2a513640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 0x7f2e240b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29511640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 0x7f2e240adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 -- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 msgr2=0x7f2e240adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 0x7f2e240adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 -- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 msgr2=0x7f2e240b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 0x7f2e240b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 -- 172.21.15.148:0/2341603487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2414d2c0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29511640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 0x7f2e240adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e2a513640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 0x7f2e240b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.489+0000 7f2e29d12640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c430 tx=0x7f2e2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.490+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200074e0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.490+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200047e0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.490+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240b62c0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.490+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20004bd0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.490+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240b5500 con 0x7f2e2406b3e0 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.491+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24074f60 con 0x7f2e2406b3e0 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.491+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2e20011070 con 0x7f2e2406b3e0 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.491+0000 7f2e1a7fc640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 0x7f2e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.491+0000 7f2e29511640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 0x7f2e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.491+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2e20089d90 con 0x7f2e2406b3e0 2024-03-04T12:13:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.492+0000 7f2e29511640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 0x7f2e0806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e240c6c10 tx=0x7f2e14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.496+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e2414d2c0 con 0x7f2e2406b3e0 2024-03-04T12:13:03.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.666+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e24072f90 con 0x7f2e2406b3e0 2024-03-04T12:13:03.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.666+0000 7f2e1a7fc640 1 -- 172.21.15.148:0/2341603487 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e20058d40 con 0x7f2e2406b3e0 2024-03-04T12:13:03.672 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 msgr2=0x7f2e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 0x7f2e0806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e240c6c10 tx=0x7f2e14009040 comp rx=0 tx=0).stop 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 msgr2=0x7f2e2414cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c430 tx=0x7f2e2000c8f0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 shutdown_connections 2024-03-04T12:13:03.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f2e0806b4d0 0x7f2e0806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e240ae030 0x7f2e240b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e240b0150 0x7f2e240adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 --2- 172.21.15.148:0/2341603487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e2406b3e0 0x7f2e2414cd80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 >> 172.21.15.148:0/2341603487 conn(0x7f2e240b22e0 msgr2=0x7f2e2406fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 shutdown_connections 2024-03-04T12:13:03.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.668+0000 7f2e2bf76640 1 -- 172.21.15.148:0/2341603487 wait complete. 2024-03-04T12:13:03.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-04T12:13:03.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.780+0000 7f7732461640 1 Processor -- start 2024-03-04T12:13:03.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.780+0000 7f7732461640 1 -- start start 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f7732461640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f772c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f7732461640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0ad560 con 0x7f772c0acda0 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f7732461640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0ad840 con 0x7f772c0cfb00 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f7732461640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0adb20 con 0x7f772c15db80 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f772b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f772c0acda0 0x7f772c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52164/0 (socket says 172.21.15.148:52164) 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f772b7fe640 1 -- 172.21.15.148:0/1016625276 learned_addr learned my addr 172.21.15.148:0/1016625276 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.781+0000 7f772bfff640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f772c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276819203 0 0) 0x7f772c0ad560 con 0x7f772c0acda0 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7714003520 con 0x7f772c0acda0 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247845011 0 0) 0x7f772c0ad840 con 0x7f772c0cfb00 2024-03-04T12:13:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c0ad560 con 0x7f772c0cfb00 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842191539 0 0) 0x7f7714003520 con 0x7f772c0acda0 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 msgr2=0x7f772c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f772c071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 >> v1:172.21.15.191:6789/0 conn(0x7f772c0cfb00 legacy=0x7f772c06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0ade00 con 0x7f772c0acda0 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f7720001670 con 0x7f772c0acda0 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 >> v1:172.21.15.148:6789/0 conn(0x7f772c0acda0 legacy=0x7f772c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f7714003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77140040d0 0x7f7714004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7714004d40 0x7f77140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0adb20 con 0x7f77140040d0 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771400baa0 con 0x7f7714004d40 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771400bca0 con 0x7f772c15db80 2024-03-04T12:13:03.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77140040d0 0x7f7714004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772bfff640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f7714003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f77309fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7714004d40 0x7f77140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 -- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 msgr2=0x7f7714003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f7714003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 -- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7714004d40 msgr2=0x7f77140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7714004d40 0x7f77140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772b7fe640 1 -- 172.21.15.148:0/1016625276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714003520 con 0x7f77140040d0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f77309fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7714004d40 0x7f77140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.782+0000 7f772bfff640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c15db80 0x7f7714003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.783+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77140040d0 0x7f7714004800 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f772000c190 tx=0x7f7714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1785bd2dbc779314 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.783+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720007cc0 con 0x7f77140040d0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.783+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720030070 con 0x7f77140040d0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.783+0000 7f772affd640 1 -- 172.21.15.148:0/1016625276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720005d70 con 0x7f77140040d0 2024-03-04T12:13:03.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 -- 172.21.15.148:0/1016625276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77140040d0 msgr2=0x7f7714004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 --2- 172.21.15.148:0/1016625276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77140040d0 0x7f7714004800 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f772000c190 tx=0x7f7714005830 comp rx=0 tx=0).stop 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f77309fe640 1 -- 172.21.15.148:0/1016625276 reap_dead start 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 -- 172.21.15.148:0/1016625276 shutdown_connections 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 -- 172.21.15.148:0/1016625276 >> 172.21.15.148:0/1016625276 conn(0x7f772c06acd0 msgr2=0x7f772c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 -- 172.21.15.148:0/1016625276 shutdown_connections 2024-03-04T12:13:03.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.784+0000 7f7732461640 1 -- 172.21.15.148:0/1016625276 wait complete. 2024-03-04T12:13:03.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.785+0000 7f7732461640 1 Processor -- start 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.785+0000 7f7732461640 1 -- start start 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77140040d0 0x7f772c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c0cd7a0 0x7f772c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c15df60 con 0x7f772c0cd320 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0ad200 con 0x7f77140040d0 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f7732461640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c071e80 con 0x7f772c0cd7a0 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53434/0 (socket says 172.21.15.148:53434) 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 -- 172.21.15.148:0/1958367731 learned_addr learned my addr 172.21.15.148:0/1958367731 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f77309fe640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c0cd7a0 0x7f772c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772bfff640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77140040d0 0x7f772c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 -- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c0cd7a0 msgr2=0x7f772c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c0cd7a0 0x7f772c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 -- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77140040d0 msgr2=0x7f772c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77140040d0 0x7f772c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 -- 172.21.15.148:0/1958367731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c15f810 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.786+0000 7f772b7fe640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f77200018f0 tx=0x7f772c0ad730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.787+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720001c00 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.787+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772002cbd0 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.787+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c15faf0 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.787+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720007070 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.787+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c1600b0 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.788+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c074f60 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.788+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7720007210 con 0x7f772c0cd320 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.788+0000 7f7728ff9640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 0x7f771806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.788+0000 7f772bfff640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 0x7f771806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.788+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f77200ab410 con 0x7f772c0cd320 2024-03-04T12:13:03.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.790+0000 7f772bfff640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 0x7f771806d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f771c0099b0 tx=0x7f771c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:03.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.793+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f772007a450 con 0x7f772c0cd320 2024-03-04T12:13:03.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.968+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f772c158100 con 0x7f772c0cd320 2024-03-04T12:13:03.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.969+0000 7f7728ff9640 1 -- 172.21.15.148:0/1958367731 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7720079d50 con 0x7f772c0cd320 2024-03-04T12:13:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:03.971 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-03-04T12:13:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-04T12:13:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-03-04T12:13:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-03-04T12:13:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-04T12:13:03.972 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-03-04T12:13:03.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.970+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 msgr2=0x7f771806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 0x7f771806d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f771c0099b0 tx=0x7f771c008040 comp rx=0 tx=0).stop 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 msgr2=0x7f772c158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f77200018f0 tx=0x7f772c0ad730 comp rx=0 tx=0).stop 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 shutdown_connections 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f771806b500 0x7f771806d9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f772c0cd7a0 0x7f772c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c0cd320 0x7f772c158900 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 --2- 172.21.15.148:0/1958367731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77140040d0 0x7f772c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 >> 172.21.15.148:0/1958367731 conn(0x7f772c06acd0 msgr2=0x7f772c06d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 shutdown_connections 2024-03-04T12:13:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:03.971+0000 7f7732461640 1 -- 172.21.15.148:0/1958367731 wait complete. 2024-03-04T12:13:03.984 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-04T12:13:03.984 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-04T12:13:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.087+0000 7f57fc8d1640 1 Processor -- start 2024-03-04T12:13:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.087+0000 7f57fc8d1640 1 -- start start 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57fc8d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57f81438e0 0x7f57f80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57fc8d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8143cc0 0x7f57f80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57fc8d1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80b2aa0 con 0x7f57f80cedc0 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57fc8d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f815e5d0 con 0x7f57f81438e0 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57fc8d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815e8b0 con 0x7f57f8143cc0 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57f81438e0 0x7f57f80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57f81438e0 0x7f57f80cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53450/0 (socket says 172.21.15.148:53450) 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 -- 172.21.15.148:0/2749019019 learned_addr learned my addr 172.21.15.148:0/2749019019 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 -- 172.21.15.148:0/2749019019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8143cc0 msgr2=0x7f57f80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57effff640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8143cc0 0x7f57f80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8143cc0 0x7f57f80ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 -- 172.21.15.148:0/2749019019 >> v1:172.21.15.191:6789/0 conn(0x7f57f80cedc0 legacy=0x7f57f8071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.088+0000 7f57f659c640 1 -- 172.21.15.148:0/2749019019 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f815eb90 con 0x7f57f81438e0 2024-03-04T12:13:04.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.089+0000 7f57f659c640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80cb910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009870 tx=0x7f57e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaf5cc57b2b38e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.089+0000 7f57f5d9b640 1 -- 172.21.15.148:0/2749019019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8035020 con 0x7f57f81438e0 2024-03-04T12:13:04.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.089+0000 7f57f5d9b640 1 -- 172.21.15.148:0/2749019019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8031640 con 0x7f57f81438e0 2024-03-04T12:13:04.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.089+0000 7f57f5d9b640 1 -- 172.21.15.148:0/2749019019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e802d040 con 0x7f57f81438e0 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 -- 172.21.15.148:0/2749019019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 msgr2=0x7f57f80cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80cb910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009870 tx=0x7f57e802b8f0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 -- 172.21.15.148:0/2749019019 shutdown_connections 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8143cc0 0x7f57f80ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/2749019019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80cb910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 -- 172.21.15.148:0/2749019019 >> 172.21.15.148:0/2749019019 conn(0x7f57f80b1600 msgr2=0x7f57f80b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 -- 172.21.15.148:0/2749019019 shutdown_connections 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.090+0000 7f57fc8d1640 1 -- 172.21.15.148:0/2749019019 wait complete. 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.091+0000 7f57fc8d1640 1 Processor -- start 2024-03-04T12:13:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.091+0000 7f57fc8d1640 1 -- start start 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f80cedc0 0x7f57f80c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f80c55e0 0x7f57f81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cf250 con 0x7f57f81438e0 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815f150 con 0x7f57f80cedc0 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57fc8d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cbe50 con 0x7f57f80c55e0 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53462/0 (socket says 172.21.15.148:53462) 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 -- 172.21.15.148:0/3230900541 learned_addr learned my addr 172.21.15.148:0/3230900541 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57f659c640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f80cedc0 0x7f57f80c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57f6d9d640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f80c55e0 0x7f57f81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 -- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f80c55e0 msgr2=0x7f57f81529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f80c55e0 0x7f57f81529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 -- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f80cedc0 msgr2=0x7f57f80c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f80cedc0 0x7f57f80c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 -- 172.21.15.148:0/3230900541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80c6b20 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57effff640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c7f0 tx=0x7f57e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0015070 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0004220 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.092+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0007620 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.093+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f80c5a80 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.093+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80c6040 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.093+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f80c6810 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.094+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f57e00079d0 con 0x7f57f81438e0 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.094+0000 7f57eeffd640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 0x7f57d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.094+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f57e008a400 con 0x7f57f81438e0 2024-03-04T12:13:04.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.094+0000 7f57f659c640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 0x7f57d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.095+0000 7f57f659c640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 0x7f57d806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57e8036000 tx=0x7f57e803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.099+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57e0059440 con 0x7f57f81438e0 2024-03-04T12:13:04.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.270+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f57f8074f60 con 0x7f57f81438e0 2024-03-04T12:13:04.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.708+0000 7f57eeffd640 1 -- 172.21.15.148:0/3230900541 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f57e0058d40 con 0x7f57f81438e0 2024-03-04T12:13:04.710 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 msgr2=0x7f57d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 0x7f57d806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57e8036000 tx=0x7f57e803c040 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 msgr2=0x7f57f80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c7f0 tx=0x7f57e000ccb0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 shutdown_connections 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f57d806b4d0 0x7f57d806d990 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57e8036000 tx=0x7f57e803c040 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f80c55e0 0x7f57f81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57f81438e0 0x7f57f80c50a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 --2- 172.21.15.148:0/3230900541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f80cedc0 0x7f57f80c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 >> 172.21.15.148:0/3230900541 conn(0x7f57f80b1600 msgr2=0x7f57f80afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 shutdown_connections 2024-03-04T12:13:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.711+0000 7f57fc8d1640 1 -- 172.21.15.148:0/3230900541 wait complete. 2024-03-04T12:13:04.727 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-04T12:13:04.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.833+0000 7efe0cf9e640 1 Processor -- start 2024-03-04T12:13:04.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.833+0000 7efe0cf9e640 1 -- start start 2024-03-04T12:13:04.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0cf9e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe0806bce0 0x7efe0806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0cf9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0cf9e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0815bae0 con 0x7efe0815b700 2024-03-04T12:13:04.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0cf9e640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0806c680 con 0x7efe0806bce0 2024-03-04T12:13:04.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0cf9e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe080adb90 con 0x7efe0815e100 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe06d9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe05d9b640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe0806bce0 0x7efe0806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0659c640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7efe0815b700 0x7efe080ae020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:39488/0 (socket says 172.21.15.148:39488) 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.834+0000 7efe0659c640 1 -- 172.21.15.148:0/612803547 learned_addr learned my addr 172.21.15.148:0/612803547 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.835+0000 7efe05d9b640 1 -- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 msgr2=0x7efe0815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.835+0000 7efe05d9b640 1 --2- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.835+0000 7efe05d9b640 1 -- 172.21.15.148:0/612803547 >> v1:172.21.15.191:6789/0 conn(0x7efe0815b700 legacy=0x7efe080ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.835+0000 7efe0559a640 1 -- 172.21.15.148:0/612803547 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297567074 0 0) 0x7efe0815bae0 con 0x7efe0815b700 2024-03-04T12:13:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.835+0000 7efe05d9b640 1 -- 172.21.15.148:0/612803547 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0815ee60 con 0x7efe0806bce0 2024-03-04T12:13:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe05d9b640 1 --2- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0806bce0 0x7efe0806c140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097a0 tx=0x7efdfc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f39528cdbd06570 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0559a640 1 -- 172.21.15.148:0/612803547 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc035020 con 0x7efe0806bce0 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0559a640 1 -- 172.21.15.148:0/612803547 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc031830 con 0x7efe0806bce0 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0559a640 1 -- 172.21.15.148:0/612803547 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc031c00 con 0x7efe0806bce0 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 -- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0806bce0 msgr2=0x7efe0806c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0806bce0 0x7efe0806c140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097a0 tx=0x7efdfc02b8d0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 -- 172.21.15.148:0/612803547 shutdown_connections 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/612803547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0806bce0 0x7efe0806c140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.836+0000 7efe0cf9e640 1 -- 172.21.15.148:0/612803547 >> 172.21.15.148:0/612803547 conn(0x7efe08071bc0 msgr2=0x7efe08071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.837+0000 7efe0cf9e640 1 -- 172.21.15.148:0/612803547 shutdown_connections 2024-03-04T12:13:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.837+0000 7efe0cf9e640 1 -- 172.21.15.148:0/612803547 wait complete. 2024-03-04T12:13:04.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.838+0000 7efe0cf9e640 1 Processor -- start 2024-03-04T12:13:04.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.838+0000 7efe0cf9e640 1 -- start start 2024-03-04T12:13:04.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 0x7efe08153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0806eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0815ead0 con 0x7efe0806bce0 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080ad820 con 0x7efe0815b700 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0815f2c0 con 0x7efe0815e100 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53466/0 (socket says 172.21.15.148:53466) 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 -- 172.21.15.148:0/1320627241 learned_addr learned my addr 172.21.15.148:0/1320627241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe05d9b640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 0x7efe08153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe06d9d640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0806eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 -- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 msgr2=0x7efe0806eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0806eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 -- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 msgr2=0x7efe08153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 0x7efe08153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 -- 172.21.15.148:0/1320627241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08153990 con 0x7efe0806bce0 2024-03-04T12:13:04.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe05d9b640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 0x7efe08153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe06d9d640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0659c640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c890 tx=0x7efdf800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8018070 con 0x7efe0806bce0 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf800a5e0 con 0x7efe0806bce0 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf800a9d0 con 0x7efe0806bce0 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe081f4190 con 0x7efe0806bce0 2024-03-04T12:13:04.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.839+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe081f4750 con 0x7efe0806bce0 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.840+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe08156ff0 con 0x7efe0806bce0 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.841+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efdf801b070 con 0x7efe0806bce0 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.841+0000 7efdf6ffd640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 0x7efde806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.841+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7efdf80897a0 con 0x7efe0806bce0 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.841+0000 7efe05d9b640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 0x7efde806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:04.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.842+0000 7efe05d9b640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 0x7efde806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdfc036000 tx=0x7efdfc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:04.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:04.846+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe08156ff0 con 0x7efe0806bce0 2024-03-04T12:13:05.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.018+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efe080cbe80 con 0x7efe0806bce0 2024-03-04T12:13:05.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.713+0000 7efdf6ffd640 1 -- 172.21.15.148:0/1320627241 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7efdf8058700 con 0x7efe0806bce0 2024-03-04T12:13:05.717 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-03-04T12:13:05.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 msgr2=0x7efde806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 0x7efde806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdfc036000 tx=0x7efdfc008030 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 msgr2=0x7efe0814fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c890 tx=0x7efdf800cd50 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 shutdown_connections 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efde806b4d0 0x7efde806d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe0815e100 0x7efe0806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe0815b700 0x7efe08153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 --2- 172.21.15.148:0/1320627241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe0806bce0 0x7efe0814fe20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 >> 172.21.15.148:0/1320627241 conn(0x7efe08071bc0 msgr2=0x7efe080b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 shutdown_connections 2024-03-04T12:13:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.715+0000 7efe0cf9e640 1 -- 172.21.15.148:0/1320627241 wait complete. 2024-03-04T12:13:05.738 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-04T12:13:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 Processor -- start 2024-03-04T12:13:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 -- start start 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f040b9410 0x7f3f041622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f040d2400 con 0x7f3f040b5700 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f040d26e0 con 0x7f3f040d34a0 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f09e97640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f040d29c0 con 0x7f3f040b9410 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.842+0000 7f3f03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f040b9410 0x7f3f041622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f037fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3f040b5700 0x7f3f040d1b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52168/0 (socket says 172.21.15.148:52168) 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f040b9410 0x7f3f041622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53474/0 (socket says 172.21.15.148:53474) 2024-03-04T12:13:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f037fe640 1 -- 172.21.15.148:0/651538308 learned_addr learned my addr 172.21.15.148:0/651538308 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f03fff640 1 -- 172.21.15.148:0/651538308 >> v1:172.21.15.191:6789/0 conn(0x7f3f040d34a0 legacy=0x7f3f04161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f03fff640 1 -- 172.21.15.148:0/651538308 >> v1:172.21.15.148:6789/0 conn(0x7f3f040b5700 legacy=0x7f3f040d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f03fff640 1 -- 172.21.15.148:0/651538308 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040d2ca0 con 0x7f3f040b9410 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.843+0000 7f3f027fc640 1 -- 172.21.15.148:0/651538308 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879153236 0 0) 0x7f3f040d2400 con 0x7f3f040b5700 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f03fff640 1 --2- 172.21.15.148:0/651538308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040b9410 0x7f3f041622f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009d90 tx=0x7f3efc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd752fcafb8961e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f027fc640 1 -- 172.21.15.148:0/651538308 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc039070 con 0x7f3f040b9410 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f027fc640 1 -- 172.21.15.148:0/651538308 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc02cc60 con 0x7f3f040b9410 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f027fc640 1 -- 172.21.15.148:0/651538308 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc034660 con 0x7f3f040b9410 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 -- 172.21.15.148:0/651538308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040b9410 msgr2=0x7f3f041622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 --2- 172.21.15.148:0/651538308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040b9410 0x7f3f041622f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009d90 tx=0x7f3efc02beb0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 -- 172.21.15.148:0/651538308 shutdown_connections 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 --2- 172.21.15.148:0/651538308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040b9410 0x7f3f041622f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 -- 172.21.15.148:0/651538308 >> 172.21.15.148:0/651538308 conn(0x7f3f040d0490 msgr2=0x7f3f040d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 -- 172.21.15.148:0/651538308 shutdown_connections 2024-03-04T12:13:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.844+0000 7f3f09e97640 1 -- 172.21.15.148:0/651538308 wait complete. 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.845+0000 7f3f09e97640 1 Processor -- start 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 -- start start 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 0x7f3f041f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f040b9410 0x7f3f041535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b5ae0 con 0x7f3f040d34a0 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04162a40 con 0x7f3f040b5700 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f09e97640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b5c60 con 0x7f3f040b9410 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53480/0 (socket says 172.21.15.148:53480) 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.846+0000 7f3f03fff640 1 -- 172.21.15.148:0/4214689442 learned_addr learned my addr 172.21.15.148:0/4214689442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f037fe640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 0x7f3f041f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f02ffd640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f040b9410 0x7f3f041535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 -- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f040b9410 msgr2=0x7f3f041535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f040b9410 0x7f3f041535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 -- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 msgr2=0x7f3f041f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 0x7f3f041f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 -- 172.21.15.148:0/4214689442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040c85b0 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f037fe640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 0x7f3f041f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f03fff640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3efc037040 tx=0x7f3efc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc039050 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc03a040 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f040c8890 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc03e610 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.847+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f040c8e50 con 0x7f3f040d34a0 2024-03-04T12:13:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.848+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f04074f60 con 0x7f3f040d34a0 2024-03-04T12:13:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.849+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3efc047020 con 0x7f3f040d34a0 2024-03-04T12:13:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.849+0000 7f3eebfff640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.849+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3efc0ac730 con 0x7f3f040d34a0 2024-03-04T12:13:05.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.849+0000 7f3f037fe640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:05.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.849+0000 7f3f037fe640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 0x7f3ef006d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:05.854+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3efc04e020 con 0x7f3f040d34a0 2024-03-04T12:13:06.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.059+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3f040b4f80 con 0x7f3f040d34a0 2024-03-04T12:13:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.723+0000 7f3eebfff640 1 -- 172.21.15.148:0/4214689442 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3efc07b540 con 0x7f3f040d34a0 2024-03-04T12:13:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 5 and data pool 6 2024-03-04T12:13:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 msgr2=0x7f3ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 0x7f3ef006d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4007dc0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 msgr2=0x7f3f0406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3efc037040 tx=0x7f3efc004a40 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 shutdown_connections 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f040d34a0 0x7f3f0406c630 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.731+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f040b9410 0x7f3f041535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.732+0000 7f3f09e97640 1 --2- 172.21.15.148:0/4214689442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f040b5700 0x7f3f041f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.732+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 >> 172.21.15.148:0/4214689442 conn(0x7f3f040d0490 msgr2=0x7f3f040d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:06.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.732+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 shutdown_connections 2024-03-04T12:13:06.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.733+0000 7f3f09e97640 1 -- 172.21.15.148:0/4214689442 wait complete. 2024-03-04T12:13:06.753 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:13:06.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.852+0000 7f82031e6640 1 Processor -- start 2024-03-04T12:13:06.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.852+0000 7f82031e6640 1 -- start start 2024-03-04T12:13:06.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f82031e6640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81fc06b3e0 0x7f81fc0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f82031e6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f82031e6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc06fd20 con 0x7f81fc15dea0 2024-03-04T12:13:06.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f82031e6640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc06fad0 con 0x7f81fc06b3e0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f82031e6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc070000 con 0x7f81fc0cc8d0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f8200f82640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f81fc15dea0 0x7f81fc06acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52174/0 (socket says 172.21.15.148:52174) 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f8200f82640 1 -- 172.21.15.148:0/846614451 learned_addr learned my addr 172.21.15.148:0/846614451 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fbfff640 1 --2- 172.21.15.148:0/846614451 >> v2:172.21.15.191:3300/0 conn(0x7f81fc06b3e0 0x7f81fc0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f8201783640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313340641 0 0) 0x7f81fc06fd20 con 0x7f81fc15dea0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec003520 con 0x7f81fc15dea0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107432609 0 0) 0x7f81ec003520 con 0x7f81fc15dea0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fbfff640 1 -- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 msgr2=0x7f81fc0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fbfff640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fbfff640 1 -- 172.21.15.148:0/846614451 >> v1:172.21.15.148:6789/0 conn(0x7f81fc15dea0 legacy=0x7f81fc06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f81fbfff640 1 -- 172.21.15.148:0/846614451 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0702e0 con 0x7f81fc06b3e0 2024-03-04T12:13:06.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.853+0000 7f8201783640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.854+0000 7f81fbfff640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc0cad60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e80097a0 tx=0x7f81e802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f6472a0b9dd5ed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.854+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8037070 con 0x7f81fc06b3e0 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.854+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8030510 con 0x7f81fc06b3e0 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.854+0000 7f81fb7fe640 1 -- 172.21.15.148:0/846614451 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0001690 con 0x7f81fc06b3e0 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 -- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 msgr2=0x7f81fc0cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc0cad60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e80097a0 tx=0x7f81e802b790 comp rx=0 tx=0).stop 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 -- 172.21.15.148:0/846614451 shutdown_connections 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 --2- 172.21.15.148:0/846614451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc0cad60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 -- 172.21.15.148:0/846614451 >> 172.21.15.148:0/846614451 conn(0x7f81fc0d0710 msgr2=0x7f81fc0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:06.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 -- 172.21.15.148:0/846614451 shutdown_connections 2024-03-04T12:13:06.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.855+0000 7f82031e6640 1 -- 172.21.15.148:0/846614451 wait complete. 2024-03-04T12:13:06.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.856+0000 7f82031e6640 1 Processor -- start 2024-03-04T12:13:06.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.856+0000 7f82031e6640 1 -- start start 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0d1380 con 0x7f81fc15dea0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0d1200 con 0x7f81fc06b3e0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f82031e6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0d1500 con 0x7f81fc0cc8d0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53488/0 (socket says 172.21.15.148:53488) 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 -- 172.21.15.148:0/2673186701 learned_addr learned my addr 172.21.15.148:0/2673186701 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8200f82640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f81fbfff640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 -- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 msgr2=0x7f81fc0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 -- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 msgr2=0x7f81fc156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 -- 172.21.15.148:0/2673186701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc156560 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.857+0000 7f8201783640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81f400efc0 tx=0x7f81f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.858+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4018030 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.858+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f400cde0 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.858+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc1f4e00 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.858+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003bb0 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.858+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc1f53c0 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.859+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc074f60 con 0x7f81fc15dea0 2024-03-04T12:13:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.859+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f81f4005d10 con 0x7f81fc15dea0 2024-03-04T12:13:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.859+0000 7f81f97fa640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 0x7f81e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.860+0000 7f8200f82640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 0x7f81e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.860+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81f408ac70 con 0x7f81fc15dea0 2024-03-04T12:13:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.860+0000 7f8200f82640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 0x7f81e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81f0001eb0 tx=0x7f81f0001e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:06.864+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81f4022070 con 0x7f81fc15dea0 2024-03-04T12:13:07.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.039+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81fc06ae50 con 0x7f81fc15dea0 2024-03-04T12:13:07.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.040+0000 7f81f97fa640 1 -- 172.21.15.148:0/2673186701 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17015 (secure 0 0 0) 0x7f81f4010040 con 0x7f81fc15dea0 2024-03-04T12:13:07.043 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:07.043 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":37,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:13:06.060354+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:13:04.288877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-04T12:13:05.037102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 msgr2=0x7f81e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 0x7f81e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81f0001eb0 tx=0x7f81f0001e40 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 msgr2=0x7f81fc0b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81f400efc0 tx=0x7f81f4009f90 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 shutdown_connections 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f81e006b4d0 0x7f81e006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81fc15dea0 0x7f81fc0b5950 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81fc0cc8d0 0x7f81fc0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 --2- 172.21.15.148:0/2673186701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81fc06b3e0 0x7f81fc156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 >> 172.21.15.148:0/2673186701 conn(0x7f81fc0d0710 msgr2=0x7f81fc0afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 shutdown_connections 2024-03-04T12:13:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.043+0000 7f82031e6640 1 -- 172.21.15.148:0/2673186701 wait complete. 2024-03-04T12:13:07.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:13:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.153+0000 7fbef79c4640 1 Processor -- start 2024-03-04T12:13:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 -- start start 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbef015de00 0x7fbef00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbef00b5470 0x7fbef00b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef015e1e0 con 0x7fbef00b9380 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef00d2820 con 0x7fbef00b5470 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.154+0000 7fbef79c4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef00d1c10 con 0x7fbef015de00 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbef00b5470 0x7fbef00b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef4f5f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbef00b9380 0x7fbef00b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52182/0 (socket says 172.21.15.148:52182) 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef4f5f640 1 -- 172.21.15.148:0/4202720289 learned_addr learned my addr 172.21.15.148:0/4202720289 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbef00b5470 0x7fbef00b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53504/0 (socket says 172.21.15.148:53504) 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 -- 172.21.15.148:0/4202720289 >> v2:172.21.15.191:3300/0 conn(0x7fbef015de00 msgr2=0x7fbef00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:07.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5760640 1 --2- 172.21.15.148:0/4202720289 >> v2:172.21.15.191:3300/0 conn(0x7fbef015de00 0x7fbef00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 --2- 172.21.15.148:0/4202720289 >> v2:172.21.15.191:3300/0 conn(0x7fbef015de00 0x7fbef00b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 -- 172.21.15.148:0/4202720289 >> v1:172.21.15.148:6789/0 conn(0x7fbef00b9380 legacy=0x7fbef00b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbeeffff640 1 -- 172.21.15.148:0/4202720289 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127788237 0 0) 0x7fbef015e1e0 con 0x7fbef00b9380 2024-03-04T12:13:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.155+0000 7fbef5f61640 1 -- 172.21.15.148:0/4202720289 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef00d1ef0 con 0x7fbef00b5470 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef5f61640 1 --2- 172.21.15.148:0/4202720289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00b5900 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ce240 tx=0x7fbee002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d54c22e1a04e4d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbeeffff640 1 -- 172.21.15.148:0/4202720289 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0035020 con 0x7fbef00b5470 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbeeffff640 1 -- 172.21.15.148:0/4202720289 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0031830 con 0x7fbef00b5470 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbeeffff640 1 -- 172.21.15.148:0/4202720289 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0031c00 con 0x7fbef00b5470 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 -- 172.21.15.148:0/4202720289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 msgr2=0x7fbef00b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 --2- 172.21.15.148:0/4202720289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00b5900 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ce240 tx=0x7fbee002b8d0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 -- 172.21.15.148:0/4202720289 shutdown_connections 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 --2- 172.21.15.148:0/4202720289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00b5900 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 --2- 172.21.15.148:0/4202720289 >> v2:172.21.15.191:3300/0 conn(0x7fbef015de00 0x7fbef00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 -- 172.21.15.148:0/4202720289 >> 172.21.15.148:0/4202720289 conn(0x7fbef00d0470 msgr2=0x7fbef00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 -- 172.21.15.148:0/4202720289 shutdown_connections 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.156+0000 7fbef79c4640 1 -- 172.21.15.148:0/4202720289 wait complete. 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 Processor -- start 2024-03-04T12:13:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 -- start start 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbef00b9380 0x7fbef00c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbef015de00 0x7fbef00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00b97e0 con 0x7fbef00b5470 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00d24b0 con 0x7fbef00b9380 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef79c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00b72a0 con 0x7fbef015de00 2024-03-04T12:13:07.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef5760640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef5760640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53508/0 (socket says 172.21.15.148:53508) 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.158+0000 7fbef5760640 1 -- 172.21.15.148:0/1618750874 learned_addr learned my addr 172.21.15.148:0/1618750874 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 -- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbef015de00 msgr2=0x7fbef00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbef015de00 0x7fbef00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 -- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbef00b9380 msgr2=0x7fbef00c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef4f5f640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbef00b9380 0x7fbef00c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbef00b9380 0x7fbef00c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 -- 172.21.15.148:0/1618750874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef00caea0 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef5760640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbee800ce40 tx=0x7fbee800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003b90 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8003d30 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef00cb180 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004240 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.159+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef00cb740 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.160+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef0074f60 con 0x7fbef00b5470 2024-03-04T12:13:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.161+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbee80043e0 con 0x7fbef00b5470 2024-03-04T12:13:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.161+0000 7fbeedffb640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 0x7fbedc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.161+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbee808aea0 con 0x7fbef00b5470 2024-03-04T12:13:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.161+0000 7fbef4f5f640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 0x7fbedc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.161+0000 7fbef4f5f640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 0x7fbedc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000f60 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.165+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbee801d340 con 0x7fbef00b5470 2024-03-04T12:13:07.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.340+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef00b37f0 con 0x7fbef00b5470 2024-03-04T12:13:07.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.341+0000 7fbeedffb640 1 -- 172.21.15.148:0/1618750874 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17015 (secure 0 0 0) 0x7fbee8059c20 con 0x7fbef00b5470 2024-03-04T12:13:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:07.344 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":37,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:13:06.060354+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:13:04.288877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-04T12:13:05.037102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:13:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 msgr2=0x7fbedc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 0x7fbedc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000f60 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2024-03-04T12:13:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 msgr2=0x7fbef00cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbee800ce40 tx=0x7fbee800a660 comp rx=0 tx=0).stop 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 shutdown_connections 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fbedc06b4d0 0x7fbedc06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbef015de00 0x7fbef00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbef00b9380 0x7fbef00c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 --2- 172.21.15.148:0/1618750874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef00b5470 0x7fbef00cbf40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 >> 172.21.15.148:0/1618750874 conn(0x7fbef00d0470 msgr2=0x7fbef00c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 shutdown_connections 2024-03-04T12:13:07.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.343+0000 7fbef79c4640 1 -- 172.21.15.148:0/1618750874 wait complete. 2024-03-04T12:13:07.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-04T12:13:07.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.459+0000 7fee114f8640 1 Processor -- start 2024-03-04T12:13:07.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.459+0000 7fee114f8640 1 -- start start 2024-03-04T12:13:07.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee114f8640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fee0c0b5c90 0x7fee0c1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee114f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee0c0b9420 0x7fee0c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee114f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0c0d3510 con 0x7fee0c15dc60 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee114f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c162720 con 0x7fee0c0b9420 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee114f8640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c0d19f0 con 0x7fee0c0b5c90 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee0c0b9420 0x7fee0c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee0c0b9420 0x7fee0c0ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53520/0 (socket says 172.21.15.148:53520) 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee0c15dc60 0x7fee0c0b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52188/0 (socket says 172.21.15.148:52188) 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 -- 172.21.15.148:0/2771266168 learned_addr learned my addr 172.21.15.148:0/2771266168 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0a7fc640 1 --2- 172.21.15.148:0/2771266168 >> v2:172.21.15.191:3300/0 conn(0x7fee0c0b5c90 0x7fee0c1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 -- 172.21.15.148:0/2771266168 >> v2:172.21.15.191:3300/0 conn(0x7fee0c0b5c90 msgr2=0x7fee0c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 --2- 172.21.15.148:0/2771266168 >> v2:172.21.15.191:3300/0 conn(0x7fee0c0b5c90 0x7fee0c1621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 -- 172.21.15.148:0/2771266168 >> v1:172.21.15.148:6789/0 conn(0x7fee0c15dc60 legacy=0x7fee0c0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee0b7fe640 1 -- 172.21.15.148:0/2771266168 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0b9920 con 0x7fee0c0b9420 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.460+0000 7fee09ffb640 1 -- 172.21.15.148:0/2771266168 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467397188 0 0) 0x7fee0c0d3510 con 0x7fee0c15dc60 2024-03-04T12:13:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee0b7fe640 1 --2- 172.21.15.148:0/2771266168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b9420 0x7fee0c0ce150 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee0c067650 tx=0x7fee0402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47a19da868a0528 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee09ffb640 1 -- 172.21.15.148:0/2771266168 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0402cb20 con 0x7fee0c0b9420 2024-03-04T12:13:07.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee09ffb640 1 -- 172.21.15.148:0/2771266168 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0402ccc0 con 0x7fee0c0b9420 2024-03-04T12:13:07.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee09ffb640 1 -- 172.21.15.148:0/2771266168 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04034610 con 0x7fee0c0b9420 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 -- 172.21.15.148:0/2771266168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b9420 msgr2=0x7fee0c0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 --2- 172.21.15.148:0/2771266168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b9420 0x7fee0c0ce150 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee0c067650 tx=0x7fee0402bf80 comp rx=0 tx=0).stop 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 -- 172.21.15.148:0/2771266168 shutdown_connections 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 --2- 172.21.15.148:0/2771266168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b9420 0x7fee0c0ce150 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 --2- 172.21.15.148:0/2771266168 >> v2:172.21.15.191:3300/0 conn(0x7fee0c0b5c90 0x7fee0c1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 -- 172.21.15.148:0/2771266168 >> 172.21.15.148:0/2771266168 conn(0x7fee0c0d0580 msgr2=0x7fee0c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 -- 172.21.15.148:0/2771266168 shutdown_connections 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.461+0000 7fee114f8640 1 -- 172.21.15.148:0/2771266168 wait complete. 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 Processor -- start 2024-03-04T12:13:07.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 -- start start 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee0c0b9420 0x7fee0c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee0c15dc60 0x7fee0c0c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0d1f20 con 0x7fee0c0b5c90 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0d2290 con 0x7fee0c15dc60 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.463+0000 7fee114f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c15e040 con 0x7fee0c0b9420 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53534/0 (socket says 172.21.15.148:53534) 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 -- 172.21.15.148:0/3158084788 learned_addr learned my addr 172.21.15.148:0/3158084788 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 -- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee0c0b9420 msgr2=0x7fee0c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee0c0b9420 0x7fee0c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 -- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee0c15dc60 msgr2=0x7fee0c0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee0c15dc60 0x7fee0c0c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 -- 172.21.15.148:0/3158084788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c1590e0 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee0affd640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00c460 tx=0x7fedfc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc018070 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc007500 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004a20 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c15a570 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.464+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c15ab30 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.465+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c074f60 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.466+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fedfc01b070 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.466+0000 7fedf37fe640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 0x7fedf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.466+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedfc089560 con 0x7fee0c0b5c90 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.466+0000 7fee0a7fc640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 0x7fedf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.466+0000 7fee0a7fc640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 0x7fedf806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee00006fd0 tx=0x7fee00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.470+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedfc0582e0 con 0x7fee0c0b5c90 2024-03-04T12:13:07.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.679+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fee0c0cba30 con 0x7fee0c0b5c90 2024-03-04T12:13:07.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.724+0000 7fedf37fe640 1 -- 172.21.15.148:0/3158084788 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fedfc057be0 con 0x7fee0c0b5c90 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 msgr2=0x7fedf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 0x7fedf806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee00006fd0 tx=0x7fee00008040 comp rx=0 tx=0).stop 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 msgr2=0x7fee0c158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00c460 tx=0x7fedfc00c920 comp rx=0 tx=0).stop 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 shutdown_connections 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fedf806b4d0 0x7fedf806d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee0c15dc60 0x7fee0c0c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee0c0b9420 0x7fee0c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 --2- 172.21.15.148:0/3158084788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b5c90 0x7fee0c158ba0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 >> 172.21.15.148:0/3158084788 conn(0x7fee0c0d0580 msgr2=0x7fee0c06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 shutdown_connections 2024-03-04T12:13:07.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.726+0000 7fee114f8640 1 -- 172.21.15.148:0/3158084788 wait complete. 2024-03-04T12:13:07.740 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-04T12:13:07.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-04T12:13:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.842+0000 7f261d737640 1 Processor -- start 2024-03-04T12:13:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.842+0000 7f261d737640 1 -- start start 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 0x7f26180d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f261806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26180b9410 0x7f261806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26180d2400 con 0x7f26180b9410 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26180d26e0 con 0x7f26180b5700 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f261d737640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180d29c0 con 0x7f26180d34a0 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26180b9410 0x7f261806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26180b9410 0x7f261806c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53548/0 (socket says 172.21.15.148:53548) 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 -- 172.21.15.148:0/3193931043 learned_addr learned my addr 172.21.15.148:0/3193931043 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f2616ffd640 1 --2- 172.21.15.148:0/3193931043 >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 0x7f26180d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f260ffff640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f261806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 -- 172.21.15.148:0/3193931043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 msgr2=0x7f261806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f261806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 -- 172.21.15.148:0/3193931043 >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 msgr2=0x7f26180d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 --2- 172.21.15.148:0/3193931043 >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 0x7f26180d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f26177fe640 1 -- 172.21.15.148:0/3193931043 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180d2ca0 con 0x7f26180b9410 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.843+0000 7f2616ffd640 1 --2- 172.21.15.148:0/3193931043 >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 0x7f26180d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.844+0000 7f26177fe640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b9410 0x7f261806c620 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f261806dad0 tx=0x7f261002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2b6c5c3103787f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.844+0000 7f26167fc640 1 -- 172.21.15.148:0/3193931043 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261002c9e0 con 0x7f26180b9410 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.844+0000 7f26167fc640 1 -- 172.21.15.148:0/3193931043 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261002cb80 con 0x7f26180b9410 2024-03-04T12:13:07.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.844+0000 7f26167fc640 1 -- 172.21.15.148:0/3193931043 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100343f0 con 0x7f26180b9410 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 -- 172.21.15.148:0/3193931043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b9410 msgr2=0x7f261806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b9410 0x7f261806c620 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f261806dad0 tx=0x7f261002be40 comp rx=0 tx=0).stop 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 -- 172.21.15.148:0/3193931043 shutdown_connections 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f261806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 --2- 172.21.15.148:0/3193931043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b9410 0x7f261806c620 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 --2- 172.21.15.148:0/3193931043 >> v2:172.21.15.191:3300/0 conn(0x7f26180b5700 0x7f26180d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 -- 172.21.15.148:0/3193931043 >> 172.21.15.148:0/3193931043 conn(0x7f26180d0490 msgr2=0x7f26180d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 -- 172.21.15.148:0/3193931043 shutdown_connections 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.845+0000 7f261d737640 1 -- 172.21.15.148:0/3193931043 wait complete. 2024-03-04T12:13:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 Processor -- start 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 -- start start 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26180b9410 0x7f2618160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f2618160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26180b5c60 con 0x7f26180b5700 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26180b5ae0 con 0x7f26180b9410 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.846+0000 7f261d737640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180b5de0 con 0x7f26180d34a0 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53564/0 (socket says 172.21.15.148:53564) 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 -- 172.21.15.148:0/3704513166 learned_addr learned my addr 172.21.15.148:0/3704513166 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f26177fe640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f2618160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f260ffff640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26180b9410 0x7f2618160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 -- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 msgr2=0x7f2618160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f2618160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 -- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26180b9410 msgr2=0x7f2618160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26180b9410 0x7f2618160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:07.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 -- 172.21.15.148:0/3704513166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261814f590 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f26177fe640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f2618160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f2616ffd640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f260800c7f0 tx=0x7f260800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608015070 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608007820 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26180cc650 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608007c10 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.847+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26180ccc10 con 0x7f26180b5700 2024-03-04T12:13:07.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.848+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618074f60 con 0x7f26180b5700 2024-03-04T12:13:07.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.849+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2608018070 con 0x7f26180b5700 2024-03-04T12:13:07.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.849+0000 7f260f7fe640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 0x7f25f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:07.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.849+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f260808a450 con 0x7f26180b5700 2024-03-04T12:13:07.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.849+0000 7f260ffff640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 0x7f25f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:07.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.850+0000 7f260ffff640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 0x7f25f806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26180c94f0 tx=0x7f2600008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:07.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:07.853+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2608059150 con 0x7f26180b5700 2024-03-04T12:13:08.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.063+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f26181616e0 con 0x7f26180b5700 2024-03-04T12:13:08.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.730+0000 7f260f7fe640 1 -- 172.21.15.148:0/3704513166 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2608058a50 con 0x7f26180b5700 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 msgr2=0x7f25f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 0x7f25f806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26180c94f0 tx=0x7f2600008040 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 msgr2=0x7f261814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f260800c7f0 tx=0x7f260800ccb0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 shutdown_connections 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f25f806b500 0x7f25f806d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26180d34a0 0x7f2618160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26180b9410 0x7f2618160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 --2- 172.21.15.148:0/3704513166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26180b5700 0x7f261814f050 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 >> 172.21.15.148:0/3704513166 conn(0x7f26180d0490 msgr2=0x7f26180ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 shutdown_connections 2024-03-04T12:13:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.733+0000 7f261d737640 1 -- 172.21.15.148:0/3704513166 wait complete. 2024-03-04T12:13:08.747 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-04T12:13:08.747 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:13:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.849+0000 7fdd84fa2640 1 Processor -- start 2024-03-04T12:13:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.849+0000 7fdd84fa2640 1 -- start start 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd84fa2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd800cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd84fa2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800afd90 con 0x7fdd8006b3e0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd84fa2640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800d0150 con 0x7fdd800cfc90 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd84fa2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006f2f0 con 0x7fdd800af9b0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd7ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdd8006b3e0 0x7fdd8015d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52192/0 (socket says 172.21.15.148:52192) 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.850+0000 7fdd7ffff640 1 -- 172.21.15.148:0/58187906 learned_addr learned my addr 172.21.15.148:0/58187906 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964358034 0 0) 0x7fdd800afd90 con 0x7fdd8006b3e0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003520 con 0x7fdd8006b3e0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd800cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795925614 0 0) 0x7fdd6c003520 con 0x7fdd8006b3e0 2024-03-04T12:13:08.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 msgr2=0x7fdd800cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd800cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 >> v1:172.21.15.191:6789/0 conn(0x7fdd800cfc90 legacy=0x7fdd8006acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80161660 con 0x7fdd8006b3e0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7fdd780019f0 con 0x7fdd8006b3e0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 >> v1:172.21.15.148:6789/0 conn(0x7fdd8006b3e0 legacy=0x7fdd8015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006f2f0 con 0x7fdd6c0039e0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c00bb50 con 0x7fdd6c004d40 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c00bd20 con 0x7fdd800af9b0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7ffff640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd76ffd640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 msgr2=0x7fdd6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd800af9b0 0x7fdd6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd6c004d40 msgr2=0x7fdd6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.851+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/58187906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c003520 con 0x7fdd6c0039e0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.852+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000ba60 tx=0x7fdd6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c189748b4867493 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.852+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70036020 con 0x7fdd6c0039e0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.852+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70032630 con 0x7fdd6c0039e0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.852+0000 7fdd7effd640 1 -- 172.21.15.148:0/58187906 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000b600 con 0x7fdd6c0039e0 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 -- 172.21.15.148:0/58187906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 msgr2=0x7fdd6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/58187906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000ba60 tx=0x7fdd6c005830 comp rx=0 tx=0).stop 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd7ffff640 1 -- 172.21.15.148:0/58187906 reap_dead start 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 -- 172.21.15.148:0/58187906 shutdown_connections 2024-03-04T12:13:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 -- 172.21.15.148:0/58187906 >> 172.21.15.148:0/58187906 conn(0x7fdd80099a00 msgr2=0x7fdd800b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 -- 172.21.15.148:0/58187906 shutdown_connections 2024-03-04T12:13:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.853+0000 7fdd84fa2640 1 -- 172.21.15.148:0/58187906 wait complete. 2024-03-04T12:13:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.854+0000 7fdd84fa2640 1 Processor -- start 2024-03-04T12:13:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.854+0000 7fdd84fa2640 1 -- start start 2024-03-04T12:13:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801e8a50 0x7fdd80149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80149eb0 0x7fdd800b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80161f00 con 0x7fdd6c0039e0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80161c20 con 0x7fdd801e8a50 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd84fa2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80162420 con 0x7fdd80149eb0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53596/0 (socket says 172.21.15.148:53596) 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/4090808370 learned_addr learned my addr 172.21.15.148:0/4090808370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7ffff640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80149eb0 0x7fdd800b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd76ffd640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801e8a50 0x7fdd80149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80149eb0 msgr2=0x7fdd800b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80149eb0 0x7fdd800b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801e8a50 msgr2=0x7fdd80149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801e8a50 0x7fdd80149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 -- 172.21.15.148:0/4090808370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd801e8fb0 con 0x7fdd6c0039e0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.855+0000 7fdd7f7fe640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002bf80 tx=0x7fdd70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.856+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70036040 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.856+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70030a40 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.856+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd801e9290 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.856+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000b600 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.856+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801e9850 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.857+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8004f100 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.857+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd70039020 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.857+0000 7fdd7cff9640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 0x7fdd6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.858+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd700acd90 con 0x7fdd6c0039e0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.858+0000 7fdd76ffd640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 0x7fdd6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.858+0000 7fdd76ffd640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 0x7fdd6006d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd801ea250 tx=0x7fdd68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:08.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:08.862+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd70045380 con 0x7fdd6c0039e0 2024-03-04T12:13:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.068+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd8006af20 con 0x7fdd6c0039e0 2024-03-04T12:13:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.069+0000 7fdd7cff9640 1 -- 172.21.15.148:0/4090808370 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fdd7007b9f0 con 0x7fdd6c0039e0 2024-03-04T12:13:09.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:09.071 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3112149074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3112149074},{"type":"v1","addr":"172.21.15.191:6835","nonce":3112149074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3334626157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3334626157},{"type":"v1","addr":"172.21.15.191:6834","nonce":3334626157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2949030514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2949030514},{"type":"v1","addr":"172.21.15.148:6836","nonce":2949030514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:13:06.060663+0000","modified":"2024-03-04T12:13:08.064737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5346":{"gid":5346,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/130422954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":130422954},{"type":"v1","addr":"172.21.15.148:6837","nonce":130422954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-04T12:13:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 15 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 msgr2=0x7fdd6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 0x7fdd6006d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd801ea250 tx=0x7fdd68009040 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 msgr2=0x7fdd801e8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002bf80 tx=0x7fdd70004730 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 shutdown_connections 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fdd6006b500 0x7fdd6006d9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80149eb0 0x7fdd800b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801e8a50 0x7fdd80149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 --2- 172.21.15.148:0/4090808370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6c0039e0 0x7fdd801e8510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 >> 172.21.15.148:0/4090808370 conn(0x7fdd80099a00 msgr2=0x7fdd800aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 shutdown_connections 2024-03-04T12:13:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.071+0000 7fdd84fa2640 1 -- 172.21.15.148:0/4090808370 wait complete. 2024-03-04T12:13:09.084 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T12:13:09.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.182+0000 7f5671844640 1 Processor -- start 2024-03-04T12:13:09.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.182+0000 7f5671844640 1 -- start start 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f5671844640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f566c0b2e60 0x7f566c0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f5671844640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f566c0b5470 0x7f566c0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f5671844640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c15e1e0 con 0x7f566c15de00 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f5671844640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0b98c0 con 0x7f566c0b2e60 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f5671844640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0b32c0 con 0x7f566c0b5470 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f566a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f566c0b2e60 0x7f566c0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f566a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f566c0b2e60 0x7f566c0b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53604/0 (socket says 172.21.15.148:53604) 2024-03-04T12:13:09.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f566affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f566c15de00 0x7f566c0b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52208/0 (socket says 172.21.15.148:52208) 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f566a7fc640 1 -- 172.21.15.148:0/2556292394 learned_addr learned my addr 172.21.15.148:0/2556292394 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.183+0000 7f566b7fe640 1 --2- 172.21.15.148:0/2556292394 >> v2:172.21.15.191:3300/0 conn(0x7f566c0b5470 0x7f566c0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f566a7fc640 1 -- 172.21.15.148:0/2556292394 >> v2:172.21.15.191:3300/0 conn(0x7f566c0b5470 msgr2=0x7f566c0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f566a7fc640 1 --2- 172.21.15.148:0/2556292394 >> v2:172.21.15.191:3300/0 conn(0x7f566c0b5470 0x7f566c0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f566a7fc640 1 -- 172.21.15.148:0/2556292394 >> v1:172.21.15.148:6789/0 conn(0x7f566c15de00 legacy=0x7f566c0b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f5669ffb640 1 -- 172.21.15.148:0/2556292394 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539800437 0 0) 0x7f566c15e1e0 con 0x7f566c15de00 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f566a7fc640 1 -- 172.21.15.148:0/2556292394 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0d1c10 con 0x7f566c0b2e60 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.184+0000 7f566a7fc640 1 --2- 172.21.15.148:0/2556292394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c0b2e60 0x7f566c0b9380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f566c0b2d40 tx=0x7f565402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ee6b3da3ccc48e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5669ffb640 1 -- 172.21.15.148:0/2556292394 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654034070 con 0x7f566c0b2e60 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5669ffb640 1 -- 172.21.15.148:0/2556292394 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654030860 con 0x7f566c0b2e60 2024-03-04T12:13:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5669ffb640 1 -- 172.21.15.148:0/2556292394 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654030c30 con 0x7f566c0b2e60 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 -- 172.21.15.148:0/2556292394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c0b2e60 msgr2=0x7f566c0b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 --2- 172.21.15.148:0/2556292394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c0b2e60 0x7f566c0b9380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f566c0b2d40 tx=0x7f565402b9a0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 -- 172.21.15.148:0/2556292394 shutdown_connections 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 --2- 172.21.15.148:0/2556292394 >> v2:172.21.15.191:3300/0 conn(0x7f566c0b5470 0x7f566c0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 --2- 172.21.15.148:0/2556292394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c0b2e60 0x7f566c0b9380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 -- 172.21.15.148:0/2556292394 >> 172.21.15.148:0/2556292394 conn(0x7f566c0d0470 msgr2=0x7f566c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 -- 172.21.15.148:0/2556292394 shutdown_connections 2024-03-04T12:13:09.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.185+0000 7f5671844640 1 -- 172.21.15.148:0/2556292394 wait complete. 2024-03-04T12:13:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.186+0000 7f5671844640 1 Processor -- start 2024-03-04T12:13:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 -- start start 2024-03-04T12:13:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 0x7f566c0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f566c0b5470 0x7f566c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0d21d0 con 0x7f566c15de00 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0d2820 con 0x7f566c0b2e60 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f5671844640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0d24b0 con 0x7f566c0b5470 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53620/0 (socket says 172.21.15.148:53620) 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.187+0000 7f566b7fe640 1 -- 172.21.15.148:0/3658980497 learned_addr learned my addr 172.21.15.148:0/3658980497 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566affd640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 0x7f566c0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566a7fc640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f566c0b5470 0x7f566c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 -- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f566c0b5470 msgr2=0x7f566c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f566c0b5470 0x7f566c0ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 -- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 msgr2=0x7f566c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 0x7f566c0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 -- 172.21.15.148:0/3658980497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0aad40 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566affd640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 0x7f566c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f566b7fe640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f566400e930 tx=0x7f566400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664017070 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566400c980 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c0aa3d0 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566400cd70 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.188+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c0ab000 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.189+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c0ccc40 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.189+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f566401a070 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.190+0000 7f5663fff640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 0x7f564c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.190+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f566408a370 con 0x7f566c15de00 2024-03-04T12:13:09.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.190+0000 7f566affd640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 0x7f564c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.190+0000 7f566affd640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 0x7f564c06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f565c007fd0 tx=0x7f565c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.194+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5664059120 con 0x7f566c15de00 2024-03-04T12:13:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.366+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f566c074f60 con 0x7f566c15de00 2024-03-04T12:13:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.367+0000 7f5663fff640 1 -- 172.21.15.148:0/3658980497 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17002 (secure 0 0 0) 0x7f5664058a20 con 0x7f566c15de00 2024-03-04T12:13:09.369 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:09.369 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T12:13:07.711884+0000","last_up_change":"2024-03-04T12:11:41.775531+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:13:04.288877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-04T12:13:05.037102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists","up"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T12:13:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 msgr2=0x7f564c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 0x7f564c06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f565c007fd0 tx=0x7f565c009040 comp rx=0 tx=0).stop 2024-03-04T12:13:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 msgr2=0x7f566c0b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f566400e930 tx=0x7f566400edf0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 shutdown_connections 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f564c06b4d0 0x7f564c06d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f566c15de00 0x7f566c0b7380 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f566c0b5470 0x7f566c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 --2- 172.21.15.148:0/3658980497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f566c0b2e60 0x7f566c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 >> 172.21.15.148:0/3658980497 conn(0x7f566c0d0470 msgr2=0x7f566c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 shutdown_connections 2024-03-04T12:13:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.369+0000 7f5671844640 1 -- 172.21.15.148:0/3658980497 wait complete. 2024-03-04T12:13:09.383 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-04T12:13:09.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:13:09.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.482+0000 7f87fb2f8640 1 Processor -- start 2024-03-04T12:13:09.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.482+0000 7f87fb2f8640 1 -- start start 2024-03-04T12:13:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87fb2f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f415bbb0 0x7f87f40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87fb2f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87f40cc690 0x7f87f40d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87fb2f8640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40cfb40 con 0x7f87f40d0e40 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87fb2f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f40d2880 con 0x7f87f40cc690 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87fb2f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40cfe20 con 0x7f87f415bbb0 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87f9895640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87f40cc690 0x7f87f40d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87f9895640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87f40cc690 0x7f87f40d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53634/0 (socket says 172.21.15.148:53634) 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.483+0000 7f87f9895640 1 -- 172.21.15.148:0/3078456503 learned_addr learned my addr 172.21.15.148:0/3078456503 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.484+0000 7f87f9094640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f415bbb0 0x7f87f40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.484+0000 7f87f9895640 1 -- 172.21.15.148:0/3078456503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f415bbb0 msgr2=0x7f87f40d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.484+0000 7f87f9895640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f415bbb0 0x7f87f40d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.484+0000 7f87f9895640 1 -- 172.21.15.148:0/3078456503 >> v1:172.21.15.191:6789/0 conn(0x7f87f40d0e40 legacy=0x7f87f40cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.484+0000 7f87f9895640 1 -- 172.21.15.148:0/3078456503 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40d0100 con 0x7f87f40cc690 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87f9895640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f40cc690 0x7f87f40d1ca0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1b49c594a72afd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87ebfff640 1 -- 172.21.15.148:0/3078456503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec034070 con 0x7f87f40cc690 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87ebfff640 1 -- 172.21.15.148:0/3078456503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec030840 con 0x7f87f40cc690 2024-03-04T12:13:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87ebfff640 1 -- 172.21.15.148:0/3078456503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec030c10 con 0x7f87f40cc690 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 -- 172.21.15.148:0/3078456503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f40cc690 msgr2=0x7f87f40d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f40cc690 0x7f87f40d1ca0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec02b980 comp rx=0 tx=0).stop 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 -- 172.21.15.148:0/3078456503 shutdown_connections 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f40cc690 0x7f87f40d1ca0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/3078456503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f415bbb0 0x7f87f40d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 -- 172.21.15.148:0/3078456503 >> 172.21.15.148:0/3078456503 conn(0x7f87f40b3480 msgr2=0x7f87f40b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 -- 172.21.15.148:0/3078456503 shutdown_connections 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.485+0000 7f87fb2f8640 1 -- 172.21.15.148:0/3078456503 wait complete. 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.486+0000 7f87fb2f8640 1 Processor -- start 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.486+0000 7f87fb2f8640 1 -- start start 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f40cc690 0x7f87f4154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f40d0e40 0x7f87f4150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415bf90 con 0x7f87f415bbb0 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40d21e0 con 0x7f87f40d0e40 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87fb2f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40ca770 con 0x7f87f40cc690 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87f9895640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87f9895640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53644/0 (socket says 172.21.15.148:53644) 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87f9895640 1 -- 172.21.15.148:0/1027851056 learned_addr learned my addr 172.21.15.148:0/1027851056 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87f9895640 1 -- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f40cc690 msgr2=0x7f87f4154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.487+0000 7f87f9895640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f40cc690 0x7f87f4154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87f9895640 1 -- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f40d0e40 msgr2=0x7f87f4150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87f8893640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f40d0e40 0x7f87f4150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87f9895640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f40d0e40 0x7f87f4150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87f9895640 1 -- 172.21.15.148:0/1027851056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40c9b60 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87f9895640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec048070 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec030640 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec034040 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f40c9e40 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.488+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f40ca400 con 0x7f87f415bbb0 2024-03-04T12:13:09.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.489+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f40b3230 con 0x7f87f415bbb0 2024-03-04T12:13:09.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.489+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f87ec04b070 con 0x7f87f415bbb0 2024-03-04T12:13:09.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.490+0000 7f87e9ffb640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 0x7f87d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.490+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87ec0ac2a0 con 0x7f87f415bbb0 2024-03-04T12:13:09.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.490+0000 7f87f9094640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 0x7f87d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.490+0000 7f87f9094640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 0x7f87d806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87f00070b0 tx=0x7f87f0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.494+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87f40b3230 con 0x7f87f415bbb0 2024-03-04T12:13:09.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.662+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87f4074f60 con 0x7f87d806b500 2024-03-04T12:13:09.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.668+0000 7f87e9ffb640 1 -- 172.21.15.148:0/1027851056 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251926 (secure 0 0 0) 0x7f87f4074f60 con 0x7f87d806b500 2024-03-04T12:13:09.673 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:09.674 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 msgr2=0x7f87d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 0x7f87d806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87f00070b0 tx=0x7f87f0007040 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 msgr2=0x7f87f41523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec004730 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 shutdown_connections 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f87d806b500 0x7f87d806d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f415bbb0 0x7f87f41523d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f40d0e40 0x7f87f4150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 --2- 172.21.15.148:0/1027851056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f40cc690 0x7f87f4154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 >> 172.21.15.148:0/1027851056 conn(0x7f87f40b3480 msgr2=0x7f87f40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 shutdown_connections 2024-03-04T12:13:09.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.671+0000 7f87fb2f8640 1 -- 172.21.15.148:0/1027851056 wait complete. 2024-03-04T12:13:09.684 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-03-04T12:13:08.729134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220776,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753944,"statfs":{"total":773094113280,"available":772868038656,"internally_reserved":0,"allocated":4661248,"data_stored":2652244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019745"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718353+0000","last_change":"2024-03-04T12:13:06.737823+0000","last_active":"2024-03-04T12:13:07.718353+0000","last_peered":"2024-03-04T12:13:07.718353+0000","last_clean":"2024-03-04T12:13:07.718353+0000","last_became_active":"2024-03-04T12:13:06.737548+0000","last_became_peered":"2024-03-04T12:13:06.737548+0000","last_unstale":"2024-03-04T12:13:07.718353+0000","last_undegraded":"2024-03-04T12:13:07.718353+0000","last_fullsized":"2024-03-04T12:13:07.718353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:38:59.796983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718177+0000","last_change":"2024-03-04T12:13:06.725044+0000","last_active":"2024-03-04T12:13:07.718177+0000","last_peered":"2024-03-04T12:13:07.718177+0000","last_clean":"2024-03-04T12:13:07.718177+0000","last_became_active":"2024-03-04T12:13:06.724776+0000","last_became_peered":"2024-03-04T12:13:06.724776+0000","last_unstale":"2024-03-04T12:13:07.718177+0000","last_undegraded":"2024-03-04T12:13:07.718177+0000","last_fullsized":"2024-03-04T12:13:07.718177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:30.998287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.737454+0000","last_change":"2024-03-04T12:13:06.734974+0000","last_active":"2024-03-04T12:13:06.737454+0000","last_peered":"2024-03-04T12:13:06.737454+0000","last_clean":"2024-03-04T12:13:06.737454+0000","last_became_active":"2024-03-04T12:13:06.734718+0000","last_became_peered":"2024-03-04T12:13:06.734718+0000","last_unstale":"2024-03-04T12:13:06.737454+0000","last_undegraded":"2024-03-04T12:13:06.737454+0000","last_fullsized":"2024-03-04T12:13:06.737454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:57.771084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731496+0000","last_change":"2024-03-04T12:13:06.746238+0000","last_active":"2024-03-04T12:13:07.731496+0000","last_peered":"2024-03-04T12:13:07.731496+0000","last_clean":"2024-03-04T12:13:07.731496+0000","last_became_active":"2024-03-04T12:13:06.745975+0000","last_became_peered":"2024-03-04T12:13:06.745975+0000","last_unstale":"2024-03-04T12:13:07.731496+0000","last_undegraded":"2024-03-04T12:13:07.731496+0000","last_fullsized":"2024-03-04T12:13:07.731496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731486+0000","last_change":"2024-03-04T12:13:06.739596+0000","last_active":"2024-03-04T12:13:07.731486+0000","last_peered":"2024-03-04T12:13:07.731486+0000","last_clean":"2024-03-04T12:13:07.731486+0000","last_became_active":"2024-03-04T12:13:06.739355+0000","last_became_peered":"2024-03-04T12:13:06.739355+0000","last_unstale":"2024-03-04T12:13:07.731486+0000","last_undegraded":"2024-03-04T12:13:07.731486+0000","last_fullsized":"2024-03-04T12:13:07.731486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739806+0000","last_change":"2024-03-04T12:13:06.739253+0000","last_active":"2024-03-04T12:13:06.739806+0000","last_peered":"2024-03-04T12:13:06.739806+0000","last_clean":"2024-03-04T12:13:06.739806+0000","last_became_active":"2024-03-04T12:13:06.738665+0000","last_became_peered":"2024-03-04T12:13:06.738665+0000","last_unstale":"2024-03-04T12:13:06.739806+0000","last_undegraded":"2024-03-04T12:13:06.739806+0000","last_fullsized":"2024-03-04T12:13:06.739806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:57.771084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719666+0000","last_change":"2024-03-04T12:13:06.724647+0000","last_active":"2024-03-04T12:13:07.719666+0000","last_peered":"2024-03-04T12:13:07.719666+0000","last_clean":"2024-03-04T12:13:07.719666+0000","last_became_active":"2024-03-04T12:13:06.724371+0000","last_became_peered":"2024-03-04T12:13:06.724371+0000","last_unstale":"2024-03-04T12:13:07.719666+0000","last_undegraded":"2024-03-04T12:13:07.719666+0000","last_fullsized":"2024-03-04T12:13:07.719666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717888+0000","last_change":"2024-03-04T12:13:06.722774+0000","last_active":"2024-03-04T12:13:07.717888+0000","last_peered":"2024-03-04T12:13:07.717888+0000","last_clean":"2024-03-04T12:13:07.717888+0000","last_became_active":"2024-03-04T12:13:06.721953+0000","last_became_peered":"2024-03-04T12:13:06.721953+0000","last_unstale":"2024-03-04T12:13:07.717888+0000","last_undegraded":"2024-03-04T12:13:07.717888+0000","last_fullsized":"2024-03-04T12:13:07.717888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736803+0000","last_change":"2024-03-04T12:13:06.733531+0000","last_active":"2024-03-04T12:13:06.736803+0000","last_peered":"2024-03-04T12:13:06.736803+0000","last_clean":"2024-03-04T12:13:06.736803+0000","last_became_active":"2024-03-04T12:13:06.733285+0000","last_became_peered":"2024-03-04T12:13:06.733285+0000","last_unstale":"2024-03-04T12:13:06.736803+0000","last_undegraded":"2024-03-04T12:13:06.736803+0000","last_fullsized":"2024-03-04T12:13:06.736803+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:14.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719861+0000","last_change":"2024-03-04T12:13:06.731707+0000","last_active":"2024-03-04T12:13:07.719861+0000","last_peered":"2024-03-04T12:13:07.719861+0000","last_clean":"2024-03-04T12:13:07.719861+0000","last_became_active":"2024-03-04T12:13:06.730341+0000","last_became_peered":"2024-03-04T12:13:06.730341+0000","last_unstale":"2024-03-04T12:13:07.719861+0000","last_undegraded":"2024-03-04T12:13:07.719861+0000","last_fullsized":"2024-03-04T12:13:07.719861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:30.998287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720076+0000","last_change":"2024-03-04T12:13:06.743427+0000","last_active":"2024-03-04T12:13:07.720076+0000","last_peered":"2024-03-04T12:13:07.720076+0000","last_clean":"2024-03-04T12:13:07.720076+0000","last_became_active":"2024-03-04T12:13:06.743163+0000","last_became_peered":"2024-03-04T12:13:06.743163+0000","last_unstale":"2024-03-04T12:13:07.720076+0000","last_undegraded":"2024-03-04T12:13:07.720076+0000","last_fullsized":"2024-03-04T12:13:07.720076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:56:04.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.731452+0000","last_change":"2024-03-04T12:13:06.727332+0000","last_active":"2024-03-04T12:13:06.731452+0000","last_peered":"2024-03-04T12:13:06.731452+0000","last_clean":"2024-03-04T12:13:06.731452+0000","last_became_active":"2024-03-04T12:13:06.727060+0000","last_became_peered":"2024-03-04T12:13:06.727060+0000","last_unstale":"2024-03-04T12:13:06.731452+0000","last_undegraded":"2024-03-04T12:13:06.731452+0000","last_fullsized":"2024-03-04T12:13:06.731452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719801+0000","last_change":"2024-03-04T12:13:06.732492+0000","last_active":"2024-03-04T12:13:07.719801+0000","last_peered":"2024-03-04T12:13:07.719801+0000","last_clean":"2024-03-04T12:13:07.719801+0000","last_became_active":"2024-03-04T12:13:06.732206+0000","last_became_peered":"2024-03-04T12:13:06.732206+0000","last_unstale":"2024-03-04T12:13:07.719801+0000","last_undegraded":"2024-03-04T12:13:07.719801+0000","last_fullsized":"2024-03-04T12:13:07.719801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745289+0000","last_change":"2024-03-04T12:13:06.743131+0000","last_active":"2024-03-04T12:13:06.745289+0000","last_peered":"2024-03-04T12:13:06.745289+0000","last_clean":"2024-03-04T12:13:06.745289+0000","last_became_active":"2024-03-04T12:13:06.742852+0000","last_became_peered":"2024-03-04T12:13:06.742852+0000","last_unstale":"2024-03-04T12:13:06.745289+0000","last_undegraded":"2024-03-04T12:13:06.745289+0000","last_fullsized":"2024-03-04T12:13:06.745289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719334+0000","last_change":"2024-03-04T12:13:06.721302+0000","last_active":"2024-03-04T12:13:07.719334+0000","last_peered":"2024-03-04T12:13:07.719334+0000","last_clean":"2024-03-04T12:13:07.719334+0000","last_became_active":"2024-03-04T12:13:06.720979+0000","last_became_peered":"2024-03-04T12:13:06.720979+0000","last_unstale":"2024-03-04T12:13:07.719334+0000","last_undegraded":"2024-03-04T12:13:07.719334+0000","last_fullsized":"2024-03-04T12:13:07.719334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.960873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718689+0000","last_change":"2024-03-04T12:13:06.738208+0000","last_active":"2024-03-04T12:13:07.718689+0000","last_peered":"2024-03-04T12:13:07.718689+0000","last_clean":"2024-03-04T12:13:07.718689+0000","last_became_active":"2024-03-04T12:13:06.737922+0000","last_became_peered":"2024-03-04T12:13:06.737922+0000","last_unstale":"2024-03-04T12:13:07.718689+0000","last_undegraded":"2024-03-04T12:13:07.718689+0000","last_fullsized":"2024-03-04T12:13:07.718689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:16:56.601953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719089+0000","last_change":"2024-03-04T12:13:06.733779+0000","last_active":"2024-03-04T12:13:07.719089+0000","last_peered":"2024-03-04T12:13:07.719089+0000","last_clean":"2024-03-04T12:13:07.719089+0000","last_became_active":"2024-03-04T12:13:06.733443+0000","last_became_peered":"2024-03-04T12:13:06.733443+0000","last_unstale":"2024-03-04T12:13:07.719089+0000","last_undegraded":"2024-03-04T12:13:07.719089+0000","last_fullsized":"2024-03-04T12:13:07.719089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:39:53.755220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749340+0000","last_change":"2024-03-04T12:13:06.746884+0000","last_active":"2024-03-04T12:13:06.749340+0000","last_peered":"2024-03-04T12:13:06.749340+0000","last_clean":"2024-03-04T12:13:06.749340+0000","last_became_active":"2024-03-04T12:13:06.746607+0000","last_became_peered":"2024-03-04T12:13:06.746607+0000","last_unstale":"2024-03-04T12:13:06.749340+0000","last_undegraded":"2024-03-04T12:13:06.749340+0000","last_fullsized":"2024-03-04T12:13:06.749340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731552+0000","last_change":"2024-03-04T12:13:06.735464+0000","last_active":"2024-03-04T12:13:07.731552+0000","last_peered":"2024-03-04T12:13:07.731552+0000","last_clean":"2024-03-04T12:13:07.731552+0000","last_became_active":"2024-03-04T12:13:06.735175+0000","last_became_peered":"2024-03-04T12:13:06.735175+0000","last_unstale":"2024-03-04T12:13:07.731552+0000","last_undegraded":"2024-03-04T12:13:07.731552+0000","last_fullsized":"2024-03-04T12:13:07.731552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747198+0000","last_change":"2024-03-04T12:13:06.746223+0000","last_active":"2024-03-04T12:13:06.747198+0000","last_peered":"2024-03-04T12:13:06.747198+0000","last_clean":"2024-03-04T12:13:06.747198+0000","last_became_active":"2024-03-04T12:13:06.745931+0000","last_became_peered":"2024-03-04T12:13:06.745931+0000","last_unstale":"2024-03-04T12:13:06.747198+0000","last_undegraded":"2024-03-04T12:13:06.747198+0000","last_fullsized":"2024-03-04T12:13:06.747198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719782+0000","last_change":"2024-03-04T12:13:06.730962+0000","last_active":"2024-03-04T12:13:07.719782+0000","last_peered":"2024-03-04T12:13:07.719782+0000","last_clean":"2024-03-04T12:13:07.719782+0000","last_became_active":"2024-03-04T12:13:06.730502+0000","last_became_peered":"2024-03-04T12:13:06.730502+0000","last_unstale":"2024-03-04T12:13:07.719782+0000","last_undegraded":"2024-03-04T12:13:07.719782+0000","last_fullsized":"2024-03-04T12:13:07.719782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747351+0000","last_change":"2024-03-04T12:13:06.745158+0000","last_active":"2024-03-04T12:13:06.747351+0000","last_peered":"2024-03-04T12:13:06.747351+0000","last_clean":"2024-03-04T12:13:06.747351+0000","last_became_active":"2024-03-04T12:13:06.744818+0000","last_became_peered":"2024-03-04T12:13:06.744818+0000","last_unstale":"2024-03-04T12:13:06.747351+0000","last_undegraded":"2024-03-04T12:13:06.747351+0000","last_fullsized":"2024-03-04T12:13:06.747351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719116+0000","last_change":"2024-03-04T12:13:06.739952+0000","last_active":"2024-03-04T12:13:07.719116+0000","last_peered":"2024-03-04T12:13:07.719116+0000","last_clean":"2024-03-04T12:13:07.719116+0000","last_became_active":"2024-03-04T12:13:06.739672+0000","last_became_peered":"2024-03-04T12:13:06.739672+0000","last_unstale":"2024-03-04T12:13:07.719116+0000","last_undegraded":"2024-03-04T12:13:07.719116+0000","last_fullsized":"2024-03-04T12:13:07.719116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:17.842475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731486+0000","last_change":"2024-03-04T12:13:06.731651+0000","last_active":"2024-03-04T12:13:07.731486+0000","last_peered":"2024-03-04T12:13:07.731486+0000","last_clean":"2024-03-04T12:13:07.731486+0000","last_became_active":"2024-03-04T12:13:06.731383+0000","last_became_peered":"2024-03-04T12:13:06.731383+0000","last_unstale":"2024-03-04T12:13:07.731486+0000","last_undegraded":"2024-03-04T12:13:07.731486+0000","last_fullsized":"2024-03-04T12:13:07.731486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739484+0000","last_change":"2024-03-04T12:13:06.738594+0000","last_active":"2024-03-04T12:13:06.739484+0000","last_peered":"2024-03-04T12:13:06.739484+0000","last_clean":"2024-03-04T12:13:06.739484+0000","last_became_active":"2024-03-04T12:13:06.738329+0000","last_became_peered":"2024-03-04T12:13:06.738329+0000","last_unstale":"2024-03-04T12:13:06.739484+0000","last_undegraded":"2024-03-04T12:13:06.739484+0000","last_fullsized":"2024-03-04T12:13:06.739484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:46.130721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.734868+0000","last_change":"2024-03-04T12:13:06.733981+0000","last_active":"2024-03-04T12:13:06.734868+0000","last_peered":"2024-03-04T12:13:06.734868+0000","last_clean":"2024-03-04T12:13:06.734868+0000","last_became_active":"2024-03-04T12:13:06.733680+0000","last_became_peered":"2024-03-04T12:13:06.733680+0000","last_unstale":"2024-03-04T12:13:06.734868+0000","last_undegraded":"2024-03-04T12:13:06.734868+0000","last_fullsized":"2024-03-04T12:13:06.734868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739221+0000","last_change":"2024-03-04T12:13:06.733781+0000","last_active":"2024-03-04T12:13:06.739221+0000","last_peered":"2024-03-04T12:13:06.739221+0000","last_clean":"2024-03-04T12:13:06.739221+0000","last_became_active":"2024-03-04T12:13:06.733527+0000","last_became_peered":"2024-03-04T12:13:06.733527+0000","last_unstale":"2024-03-04T12:13:06.739221+0000","last_undegraded":"2024-03-04T12:13:06.739221+0000","last_fullsized":"2024-03-04T12:13:06.739221+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:55.200883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720083+0000","last_change":"2024-03-04T12:13:06.720879+0000","last_active":"2024-03-04T12:13:07.720083+0000","last_peered":"2024-03-04T12:13:07.720083+0000","last_clean":"2024-03-04T12:13:07.720083+0000","last_became_active":"2024-03-04T12:13:06.720605+0000","last_became_peered":"2024-03-04T12:13:06.720605+0000","last_unstale":"2024-03-04T12:13:07.720083+0000","last_undegraded":"2024-03-04T12:13:07.720083+0000","last_fullsized":"2024-03-04T12:13:07.720083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:53:28.474198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739283+0000","last_change":"2024-03-04T12:13:06.733427+0000","last_active":"2024-03-04T12:13:06.739283+0000","last_peered":"2024-03-04T12:13:06.739283+0000","last_clean":"2024-03-04T12:13:06.739283+0000","last_became_active":"2024-03-04T12:13:06.733122+0000","last_became_peered":"2024-03-04T12:13:06.733122+0000","last_unstale":"2024-03-04T12:13:06.739283+0000","last_undegraded":"2024-03-04T12:13:06.739283+0000","last_fullsized":"2024-03-04T12:13:06.739283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:57.298861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739346+0000","last_change":"2024-03-04T12:13:06.737090+0000","last_active":"2024-03-04T12:13:06.739346+0000","last_peered":"2024-03-04T12:13:06.739346+0000","last_clean":"2024-03-04T12:13:06.739346+0000","last_became_active":"2024-03-04T12:13:06.736832+0000","last_became_peered":"2024-03-04T12:13:06.736832+0000","last_unstale":"2024-03-04T12:13:06.739346+0000","last_undegraded":"2024-03-04T12:13:06.739346+0000","last_fullsized":"2024-03-04T12:13:06.739346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:07:37.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719208+0000","last_change":"2024-03-04T12:13:06.735954+0000","last_active":"2024-03-04T12:13:07.719208+0000","last_peered":"2024-03-04T12:13:07.719208+0000","last_clean":"2024-03-04T12:13:07.719208+0000","last_became_active":"2024-03-04T12:13:06.735672+0000","last_became_peered":"2024-03-04T12:13:06.735672+0000","last_unstale":"2024-03-04T12:13:07.719208+0000","last_undegraded":"2024-03-04T12:13:07.719208+0000","last_fullsized":"2024-03-04T12:13:07.719208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.960873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.731552+0000","last_change":"2024-03-04T12:13:06.730175+0000","last_active":"2024-03-04T12:13:06.731552+0000","last_peered":"2024-03-04T12:13:06.731552+0000","last_clean":"2024-03-04T12:13:06.731552+0000","last_became_active":"2024-03-04T12:13:06.729927+0000","last_became_peered":"2024-03-04T12:13:06.729927+0000","last_unstale":"2024-03-04T12:13:06.731552+0000","last_undegraded":"2024-03-04T12:13:06.731552+0000","last_fullsized":"2024-03-04T12:13:06.731552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719373+0000","last_change":"2024-03-04T12:13:06.735462+0000","last_active":"2024-03-04T12:13:07.719373+0000","last_peered":"2024-03-04T12:13:07.719373+0000","last_clean":"2024-03-04T12:13:07.719373+0000","last_became_active":"2024-03-04T12:13:06.734980+0000","last_became_peered":"2024-03-04T12:13:06.734980+0000","last_unstale":"2024-03-04T12:13:07.719373+0000","last_undegraded":"2024-03-04T12:13:07.719373+0000","last_fullsized":"2024-03-04T12:13:07.719373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:12:58.923163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747940+0000","last_change":"2024-03-04T12:13:05.736181+0000","last_active":"2024-03-04T12:13:06.747940+0000","last_peered":"2024-03-04T12:13:06.747940+0000","last_clean":"2024-03-04T12:13:06.747940+0000","last_became_active":"2024-03-04T12:13:05.735906+0000","last_became_peered":"2024-03-04T12:13:05.735906+0000","last_unstale":"2024-03-04T12:13:06.747940+0000","last_undegraded":"2024-03-04T12:13:06.747940+0000","last_fullsized":"2024-03-04T12:13:06.747940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:52:54.955194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.743516+0000","last_change":"2024-03-04T12:13:06.742364+0000","last_active":"2024-03-04T12:13:06.743516+0000","last_peered":"2024-03-04T12:13:06.743516+0000","last_clean":"2024-03-04T12:13:06.743516+0000","last_became_active":"2024-03-04T12:13:06.742108+0000","last_became_peered":"2024-03-04T12:13:06.742108+0000","last_unstale":"2024-03-04T12:13:06.743516+0000","last_undegraded":"2024-03-04T12:13:06.743516+0000","last_fullsized":"2024-03-04T12:13:06.743516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:57.516181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749320+0000","last_change":"2024-03-04T12:13:05.735007+0000","last_active":"2024-03-04T12:13:06.749320+0000","last_peered":"2024-03-04T12:13:06.749320+0000","last_clean":"2024-03-04T12:13:06.749320+0000","last_became_active":"2024-03-04T12:13:05.734294+0000","last_became_peered":"2024-03-04T12:13:05.734294+0000","last_unstale":"2024-03-04T12:13:06.749320+0000","last_undegraded":"2024-03-04T12:13:06.749320+0000","last_fullsized":"2024-03-04T12:13:06.749320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:38.435861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.742621+0000","last_change":"2024-03-04T12:13:06.739556+0000","last_active":"2024-03-04T12:13:06.742621+0000","last_peered":"2024-03-04T12:13:06.742621+0000","last_clean":"2024-03-04T12:13:06.742621+0000","last_became_active":"2024-03-04T12:13:06.739113+0000","last_became_peered":"2024-03-04T12:13:06.739113+0000","last_unstale":"2024-03-04T12:13:06.742621+0000","last_undegraded":"2024-03-04T12:13:06.742621+0000","last_fullsized":"2024-03-04T12:13:06.742621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:46.130721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.742858+0000","last_change":"2024-03-04T12:13:05.736255+0000","last_active":"2024-03-04T12:13:06.742858+0000","last_peered":"2024-03-04T12:13:06.742858+0000","last_clean":"2024-03-04T12:13:06.742858+0000","last_became_active":"2024-03-04T12:13:05.735991+0000","last_became_peered":"2024-03-04T12:13:05.735991+0000","last_unstale":"2024-03-04T12:13:06.742858+0000","last_undegraded":"2024-03-04T12:13:06.742858+0000","last_fullsized":"2024-03-04T12:13:06.742858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:39.246078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718337+0000","last_change":"2024-03-04T12:13:06.735687+0000","last_active":"2024-03-04T12:13:07.718337+0000","last_peered":"2024-03-04T12:13:07.718337+0000","last_clean":"2024-03-04T12:13:07.718337+0000","last_became_active":"2024-03-04T12:13:06.735409+0000","last_became_peered":"2024-03-04T12:13:06.735409+0000","last_unstale":"2024-03-04T12:13:07.718337+0000","last_undegraded":"2024-03-04T12:13:07.718337+0000","last_fullsized":"2024-03-04T12:13:07.718337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:00.391256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745284+0000","last_change":"2024-03-04T12:13:05.726597+0000","last_active":"2024-03-04T12:13:06.745284+0000","last_peered":"2024-03-04T12:13:06.745284+0000","last_clean":"2024-03-04T12:13:06.745284+0000","last_became_active":"2024-03-04T12:13:05.726290+0000","last_became_peered":"2024-03-04T12:13:05.726290+0000","last_unstale":"2024-03-04T12:13:06.745284+0000","last_undegraded":"2024-03-04T12:13:06.745284+0000","last_fullsized":"2024-03-04T12:13:06.745284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:07:36.898994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736165+0000","last_change":"2024-03-04T12:13:06.730947+0000","last_active":"2024-03-04T12:13:06.736165+0000","last_peered":"2024-03-04T12:13:06.736165+0000","last_clean":"2024-03-04T12:13:06.736165+0000","last_became_active":"2024-03-04T12:13:06.730684+0000","last_became_peered":"2024-03-04T12:13:06.730684+0000","last_unstale":"2024-03-04T12:13:06.736165+0000","last_undegraded":"2024-03-04T12:13:06.736165+0000","last_fullsized":"2024-03-04T12:13:06.736165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719936+0000","last_change":"2024-03-04T12:13:05.736036+0000","last_active":"2024-03-04T12:13:07.719936+0000","last_peered":"2024-03-04T12:13:07.719936+0000","last_clean":"2024-03-04T12:13:07.719936+0000","last_became_active":"2024-03-04T12:13:05.735708+0000","last_became_peered":"2024-03-04T12:13:05.735708+0000","last_unstale":"2024-03-04T12:13:07.719936+0000","last_undegraded":"2024-03-04T12:13:07.719936+0000","last_fullsized":"2024-03-04T12:13:07.719936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:44.902387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719526+0000","last_change":"2024-03-04T12:13:06.725433+0000","last_active":"2024-03-04T12:13:07.719526+0000","last_peered":"2024-03-04T12:13:07.719526+0000","last_clean":"2024-03-04T12:13:07.719526+0000","last_became_active":"2024-03-04T12:13:06.725144+0000","last_became_peered":"2024-03-04T12:13:06.725144+0000","last_unstale":"2024-03-04T12:13:07.719526+0000","last_undegraded":"2024-03-04T12:13:07.719526+0000","last_fullsized":"2024-03-04T12:13:07.719526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:51.258924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719081+0000","last_change":"2024-03-04T12:13:05.724819+0000","last_active":"2024-03-04T12:13:07.719081+0000","last_peered":"2024-03-04T12:13:07.719081+0000","last_clean":"2024-03-04T12:13:07.719081+0000","last_became_active":"2024-03-04T12:13:05.724561+0000","last_became_peered":"2024-03-04T12:13:05.724561+0000","last_unstale":"2024-03-04T12:13:07.719081+0000","last_undegraded":"2024-03-04T12:13:07.719081+0000","last_fullsized":"2024-03-04T12:13:07.719081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:44.902387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745010+0000","last_change":"2024-03-04T12:13:06.740150+0000","last_active":"2024-03-04T12:13:06.745010+0000","last_peered":"2024-03-04T12:13:06.745010+0000","last_clean":"2024-03-04T12:13:06.745010+0000","last_became_active":"2024-03-04T12:13:06.739502+0000","last_became_peered":"2024-03-04T12:13:06.739502+0000","last_unstale":"2024-03-04T12:13:06.745010+0000","last_undegraded":"2024-03-04T12:13:06.745010+0000","last_fullsized":"2024-03-04T12:13:06.745010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725514+0000","last_change":"2024-03-04T12:13:05.733273+0000","last_active":"2024-03-04T12:13:06.725514+0000","last_peered":"2024-03-04T12:13:06.725514+0000","last_clean":"2024-03-04T12:13:06.725514+0000","last_became_active":"2024-03-04T12:13:05.733005+0000","last_became_peered":"2024-03-04T12:13:05.733005+0000","last_unstale":"2024-03-04T12:13:06.725514+0000","last_undegraded":"2024-03-04T12:13:06.725514+0000","last_fullsized":"2024-03-04T12:13:06.725514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:45.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718171+0000","last_change":"2024-03-04T12:13:06.732782+0000","last_active":"2024-03-04T12:13:07.718171+0000","last_peered":"2024-03-04T12:13:07.718171+0000","last_clean":"2024-03-04T12:13:07.718171+0000","last_became_active":"2024-03-04T12:13:06.732451+0000","last_became_peered":"2024-03-04T12:13:06.732451+0000","last_unstale":"2024-03-04T12:13:07.718171+0000","last_undegraded":"2024-03-04T12:13:07.718171+0000","last_fullsized":"2024-03-04T12:13:07.718171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:45:48.052770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725723+0000","last_change":"2024-03-04T12:13:05.731307+0000","last_active":"2024-03-04T12:13:06.725723+0000","last_peered":"2024-03-04T12:13:06.725723+0000","last_clean":"2024-03-04T12:13:06.725723+0000","last_became_active":"2024-03-04T12:13:05.731030+0000","last_became_peered":"2024-03-04T12:13:05.731030+0000","last_unstale":"2024-03-04T12:13:06.725723+0000","last_undegraded":"2024-03-04T12:13:06.725723+0000","last_fullsized":"2024-03-04T12:13:06.725723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:56.297011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731503+0000","last_change":"2024-03-04T12:13:06.746689+0000","last_active":"2024-03-04T12:13:07.731503+0000","last_peered":"2024-03-04T12:13:07.731503+0000","last_clean":"2024-03-04T12:13:07.731503+0000","last_became_active":"2024-03-04T12:13:06.746409+0000","last_became_peered":"2024-03-04T12:13:06.746409+0000","last_unstale":"2024-03-04T12:13:07.731503+0000","last_undegraded":"2024-03-04T12:13:07.731503+0000","last_fullsized":"2024-03-04T12:13:07.731503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718424+0000","last_change":"2024-03-04T12:13:05.728812+0000","last_active":"2024-03-04T12:13:07.718424+0000","last_peered":"2024-03-04T12:13:07.718424+0000","last_clean":"2024-03-04T12:13:07.718424+0000","last_became_active":"2024-03-04T12:13:05.728535+0000","last_became_peered":"2024-03-04T12:13:05.728535+0000","last_unstale":"2024-03-04T12:13:07.718424+0000","last_undegraded":"2024-03-04T12:13:07.718424+0000","last_fullsized":"2024-03-04T12:13:07.718424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:21:49.594403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719469+0000","last_change":"2024-03-04T12:13:06.738097+0000","last_active":"2024-03-04T12:13:07.719469+0000","last_peered":"2024-03-04T12:13:07.719469+0000","last_clean":"2024-03-04T12:13:07.719469+0000","last_became_active":"2024-03-04T12:13:06.737820+0000","last_became_peered":"2024-03-04T12:13:06.737820+0000","last_unstale":"2024-03-04T12:13:07.719469+0000","last_undegraded":"2024-03-04T12:13:07.719469+0000","last_fullsized":"2024-03-04T12:13:07.719469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719995+0000","last_change":"2024-03-04T12:13:05.730546+0000","last_active":"2024-03-04T12:13:07.719995+0000","last_peered":"2024-03-04T12:13:07.719995+0000","last_clean":"2024-03-04T12:13:07.719995+0000","last_became_active":"2024-03-04T12:13:05.729638+0000","last_became_peered":"2024-03-04T12:13:05.729638+0000","last_unstale":"2024-03-04T12:13:07.719995+0000","last_undegraded":"2024-03-04T12:13:07.719995+0000","last_fullsized":"2024-03-04T12:13:07.719995+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:56.769234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.738230+0000","last_change":"2024-03-04T12:13:06.737086+0000","last_active":"2024-03-04T12:13:06.738230+0000","last_peered":"2024-03-04T12:13:06.738230+0000","last_clean":"2024-03-04T12:13:06.738230+0000","last_became_active":"2024-03-04T12:13:06.736813+0000","last_became_peered":"2024-03-04T12:13:06.736813+0000","last_unstale":"2024-03-04T12:13:06.738230+0000","last_undegraded":"2024-03-04T12:13:06.738230+0000","last_fullsized":"2024-03-04T12:13:06.738230+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:46.393815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720140+0000","last_change":"2024-03-04T12:13:05.731299+0000","last_active":"2024-03-04T12:13:07.720140+0000","last_peered":"2024-03-04T12:13:07.720140+0000","last_clean":"2024-03-04T12:13:07.720140+0000","last_became_active":"2024-03-04T12:13:05.730996+0000","last_became_peered":"2024-03-04T12:13:05.730996+0000","last_unstale":"2024-03-04T12:13:07.720140+0000","last_undegraded":"2024-03-04T12:13:07.720140+0000","last_fullsized":"2024-03-04T12:13:07.720140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:13.739896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747305+0000","last_change":"2024-03-04T12:13:06.744627+0000","last_active":"2024-03-04T12:13:06.747305+0000","last_peered":"2024-03-04T12:13:06.747305+0000","last_clean":"2024-03-04T12:13:06.747305+0000","last_became_active":"2024-03-04T12:13:06.744368+0000","last_became_peered":"2024-03-04T12:13:06.744368+0000","last_unstale":"2024-03-04T12:13:06.747305+0000","last_undegraded":"2024-03-04T12:13:06.747305+0000","last_fullsized":"2024-03-04T12:13:06.747305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718299+0000","last_change":"2024-03-04T12:13:05.725302+0000","last_active":"2024-03-04T12:13:07.718299+0000","last_peered":"2024-03-04T12:13:07.718299+0000","last_clean":"2024-03-04T12:13:07.718299+0000","last_became_active":"2024-03-04T12:13:05.725040+0000","last_became_peered":"2024-03-04T12:13:05.725040+0000","last_unstale":"2024-03-04T12:13:07.718299+0000","last_undegraded":"2024-03-04T12:13:07.718299+0000","last_fullsized":"2024-03-04T12:13:07.718299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:40:46.456139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736827+0000","last_change":"2024-03-04T12:13:06.733377+0000","last_active":"2024-03-04T12:13:06.736827+0000","last_peered":"2024-03-04T12:13:06.736827+0000","last_clean":"2024-03-04T12:13:06.736827+0000","last_became_active":"2024-03-04T12:13:06.733101+0000","last_became_peered":"2024-03-04T12:13:06.733101+0000","last_unstale":"2024-03-04T12:13:06.736827+0000","last_undegraded":"2024-03-04T12:13:06.736827+0000","last_fullsized":"2024-03-04T12:13:06.736827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718298+0000","last_change":"2024-03-04T12:13:05.734422+0000","last_active":"2024-03-04T12:13:07.718298+0000","last_peered":"2024-03-04T12:13:07.718298+0000","last_clean":"2024-03-04T12:13:07.718298+0000","last_became_active":"2024-03-04T12:13:05.734160+0000","last_became_peered":"2024-03-04T12:13:05.734160+0000","last_unstale":"2024-03-04T12:13:07.718298+0000","last_undegraded":"2024-03-04T12:13:07.718298+0000","last_fullsized":"2024-03-04T12:13:07.718298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:54.000013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717873+0000","last_change":"2024-03-04T12:13:06.730358+0000","last_active":"2024-03-04T12:13:07.717873+0000","last_peered":"2024-03-04T12:13:07.717873+0000","last_clean":"2024-03-04T12:13:07.717873+0000","last_became_active":"2024-03-04T12:13:06.730109+0000","last_became_peered":"2024-03-04T12:13:06.730109+0000","last_unstale":"2024-03-04T12:13:07.717873+0000","last_undegraded":"2024-03-04T12:13:07.717873+0000","last_fullsized":"2024-03-04T12:13:07.717873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725660+0000","last_change":"2024-03-04T12:13:05.732806+0000","last_active":"2024-03-04T12:13:06.725660+0000","last_peered":"2024-03-04T12:13:06.725660+0000","last_clean":"2024-03-04T12:13:06.725660+0000","last_became_active":"2024-03-04T12:13:05.732567+0000","last_became_peered":"2024-03-04T12:13:05.732567+0000","last_unstale":"2024-03-04T12:13:06.725660+0000","last_undegraded":"2024-03-04T12:13:06.725660+0000","last_fullsized":"2024-03-04T12:13:06.725660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:38.435861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718832+0000","last_change":"2024-03-04T12:13:06.736969+0000","last_active":"2024-03-04T12:13:07.718832+0000","last_peered":"2024-03-04T12:13:07.718832+0000","last_clean":"2024-03-04T12:13:07.718832+0000","last_became_active":"2024-03-04T12:13:06.736682+0000","last_became_peered":"2024-03-04T12:13:06.736682+0000","last_unstale":"2024-03-04T12:13:07.718832+0000","last_undegraded":"2024-03-04T12:13:07.718832+0000","last_fullsized":"2024-03-04T12:13:07.718832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:11.380195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.724993+0000","last_change":"2024-03-04T12:13:05.738768+0000","last_active":"2024-03-04T12:13:06.724993+0000","last_peered":"2024-03-04T12:13:06.724993+0000","last_clean":"2024-03-04T12:13:06.724993+0000","last_became_active":"2024-03-04T12:13:05.738446+0000","last_became_peered":"2024-03-04T12:13:05.738446+0000","last_unstale":"2024-03-04T12:13:06.724993+0000","last_undegraded":"2024-03-04T12:13:06.724993+0000","last_fullsized":"2024-03-04T12:13:06.724993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:01:05.209750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718094+0000","last_change":"2024-03-04T12:13:06.729755+0000","last_active":"2024-03-04T12:13:07.718094+0000","last_peered":"2024-03-04T12:13:07.718094+0000","last_clean":"2024-03-04T12:13:07.718094+0000","last_became_active":"2024-03-04T12:13:06.729490+0000","last_became_peered":"2024-03-04T12:13:06.729490+0000","last_unstale":"2024-03-04T12:13:07.718094+0000","last_undegraded":"2024-03-04T12:13:07.718094+0000","last_fullsized":"2024-03-04T12:13:07.718094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718810+0000","last_change":"2024-03-04T12:13:05.724337+0000","last_active":"2024-03-04T12:13:07.718810+0000","last_peered":"2024-03-04T12:13:07.718810+0000","last_clean":"2024-03-04T12:13:07.718810+0000","last_became_active":"2024-03-04T12:13:05.724047+0000","last_became_peered":"2024-03-04T12:13:05.724047+0000","last_unstale":"2024-03-04T12:13:07.718810+0000","last_undegraded":"2024-03-04T12:13:07.718810+0000","last_fullsized":"2024-03-04T12:13:07.718810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:24.459143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718309+0000","last_change":"2024-03-04T12:13:06.736490+0000","last_active":"2024-03-04T12:13:07.718309+0000","last_peered":"2024-03-04T12:13:07.718309+0000","last_clean":"2024-03-04T12:13:07.718309+0000","last_became_active":"2024-03-04T12:13:06.735977+0000","last_became_peered":"2024-03-04T12:13:06.735977+0000","last_unstale":"2024-03-04T12:13:07.718309+0000","last_undegraded":"2024-03-04T12:13:07.718309+0000","last_fullsized":"2024-03-04T12:13:07.718309+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:56:04.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720024+0000","last_change":"2024-03-04T12:13:05.736454+0000","last_active":"2024-03-04T12:13:07.720024+0000","last_peered":"2024-03-04T12:13:07.720024+0000","last_clean":"2024-03-04T12:13:07.720024+0000","last_became_active":"2024-03-04T12:13:05.736151+0000","last_became_peered":"2024-03-04T12:13:05.736151+0000","last_unstale":"2024-03-04T12:13:07.720024+0000","last_undegraded":"2024-03-04T12:13:07.720024+0000","last_fullsized":"2024-03-04T12:13:07.720024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:45.391965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719989+0000","last_change":"2024-03-04T12:13:06.731886+0000","last_active":"2024-03-04T12:13:07.719989+0000","last_peered":"2024-03-04T12:13:07.719989+0000","last_clean":"2024-03-04T12:13:07.719989+0000","last_became_active":"2024-03-04T12:13:06.731577+0000","last_became_peered":"2024-03-04T12:13:06.731577+0000","last_unstale":"2024-03-04T12:13:07.719989+0000","last_undegraded":"2024-03-04T12:13:07.719989+0000","last_fullsized":"2024-03-04T12:13:07.719989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:25.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749826+0000","last_change":"2024-03-04T12:13:05.732482+0000","last_active":"2024-03-04T12:13:06.749826+0000","last_peered":"2024-03-04T12:13:06.749826+0000","last_clean":"2024-03-04T12:13:06.749826+0000","last_became_active":"2024-03-04T12:13:05.732195+0000","last_became_peered":"2024-03-04T12:13:05.732195+0000","last_unstale":"2024-03-04T12:13:06.749826+0000","last_undegraded":"2024-03-04T12:13:06.749826+0000","last_fullsized":"2024-03-04T12:13:06.749826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:34:59.445308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.743494+0000","last_change":"2024-03-04T12:13:06.740765+0000","last_active":"2024-03-04T12:13:06.743494+0000","last_peered":"2024-03-04T12:13:06.743494+0000","last_clean":"2024-03-04T12:13:06.743494+0000","last_became_active":"2024-03-04T12:13:06.740498+0000","last_became_peered":"2024-03-04T12:13:06.740498+0000","last_unstale":"2024-03-04T12:13:06.743494+0000","last_undegraded":"2024-03-04T12:13:06.743494+0000","last_fullsized":"2024-03-04T12:13:06.743494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731597+0000","last_change":"2024-03-04T12:13:05.731193+0000","last_active":"2024-03-04T12:13:07.731597+0000","last_peered":"2024-03-04T12:13:07.731597+0000","last_clean":"2024-03-04T12:13:07.731597+0000","last_became_active":"2024-03-04T12:13:05.730883+0000","last_became_peered":"2024-03-04T12:13:05.730883+0000","last_unstale":"2024-03-04T12:13:07.731597+0000","last_undegraded":"2024-03-04T12:13:07.731597+0000","last_fullsized":"2024-03-04T12:13:07.731597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:01:05.209750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731698+0000","last_change":"2024-03-04T12:13:06.724271+0000","last_active":"2024-03-04T12:13:07.731698+0000","last_peered":"2024-03-04T12:13:07.731698+0000","last_clean":"2024-03-04T12:13:07.731698+0000","last_became_active":"2024-03-04T12:13:06.723942+0000","last_became_peered":"2024-03-04T12:13:06.723942+0000","last_unstale":"2024-03-04T12:13:07.731698+0000","last_undegraded":"2024-03-04T12:13:07.731698+0000","last_fullsized":"2024-03-04T12:13:07.731698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.728594+0000","last_change":"2024-03-04T12:13:05.736760+0000","last_active":"2024-03-04T12:13:06.728594+0000","last_peered":"2024-03-04T12:13:06.728594+0000","last_clean":"2024-03-04T12:13:06.728594+0000","last_became_active":"2024-03-04T12:13:05.736432+0000","last_became_peered":"2024-03-04T12:13:05.736432+0000","last_unstale":"2024-03-04T12:13:06.728594+0000","last_undegraded":"2024-03-04T12:13:06.728594+0000","last_fullsized":"2024-03-04T12:13:06.728594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:24.459143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719412+0000","last_change":"2024-03-04T12:13:06.731814+0000","last_active":"2024-03-04T12:13:07.719412+0000","last_peered":"2024-03-04T12:13:07.719412+0000","last_clean":"2024-03-04T12:13:07.719412+0000","last_became_active":"2024-03-04T12:13:06.731540+0000","last_became_peered":"2024-03-04T12:13:06.731540+0000","last_unstale":"2024-03-04T12:13:07.719412+0000","last_undegraded":"2024-03-04T12:13:07.719412+0000","last_fullsized":"2024-03-04T12:13:07.719412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731515+0000","last_change":"2024-03-04T12:13:05.736704+0000","last_active":"2024-03-04T12:13:07.731515+0000","last_peered":"2024-03-04T12:13:07.731515+0000","last_clean":"2024-03-04T12:13:07.731515+0000","last_became_active":"2024-03-04T12:13:05.735570+0000","last_became_peered":"2024-03-04T12:13:05.735570+0000","last_unstale":"2024-03-04T12:13:07.731515+0000","last_undegraded":"2024-03-04T12:13:07.731515+0000","last_fullsized":"2024-03-04T12:13:07.731515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.788131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736761+0000","last_change":"2024-03-04T12:13:06.735878+0000","last_active":"2024-03-04T12:13:06.736761+0000","last_peered":"2024-03-04T12:13:06.736761+0000","last_clean":"2024-03-04T12:13:06.736761+0000","last_became_active":"2024-03-04T12:13:06.735629+0000","last_became_peered":"2024-03-04T12:13:06.735629+0000","last_unstale":"2024-03-04T12:13:06.736761+0000","last_undegraded":"2024-03-04T12:13:06.736761+0000","last_fullsized":"2024-03-04T12:13:06.736761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:57.516181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731375+0000","last_change":"2024-03-04T12:13:05.738609+0000","last_active":"2024-03-04T12:13:07.731375+0000","last_peered":"2024-03-04T12:13:07.731375+0000","last_clean":"2024-03-04T12:13:07.731375+0000","last_became_active":"2024-03-04T12:13:05.738411+0000","last_became_peered":"2024-03-04T12:13:05.738411+0000","last_unstale":"2024-03-04T12:13:07.731375+0000","last_undegraded":"2024-03-04T12:13:07.731375+0000","last_fullsized":"2024-03-04T12:13:07.731375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:26:25.947943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719736+0000","last_change":"2024-03-04T12:13:06.738677+0000","last_active":"2024-03-04T12:13:07.719736+0000","last_peered":"2024-03-04T12:13:07.719736+0000","last_clean":"2024-03-04T12:13:07.719736+0000","last_became_active":"2024-03-04T12:13:06.737413+0000","last_became_peered":"2024-03-04T12:13:06.737413+0000","last_unstale":"2024-03-04T12:13:07.719736+0000","last_undegraded":"2024-03-04T12:13:07.719736+0000","last_fullsized":"2024-03-04T12:13:07.719736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:12:58.923163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719286+0000","last_change":"2024-03-04T12:13:05.731210+0000","last_active":"2024-03-04T12:13:07.719286+0000","last_peered":"2024-03-04T12:13:07.719286+0000","last_clean":"2024-03-04T12:13:07.719286+0000","last_became_active":"2024-03-04T12:13:05.730892+0000","last_became_peered":"2024-03-04T12:13:05.730892+0000","last_unstale":"2024-03-04T12:13:07.719286+0000","last_undegraded":"2024-03-04T12:13:07.719286+0000","last_fullsized":"2024-03-04T12:13:07.719286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:29.358867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718243+0000","last_change":"2024-03-04T12:13:06.733212+0000","last_active":"2024-03-04T12:13:07.718243+0000","last_peered":"2024-03-04T12:13:07.718243+0000","last_clean":"2024-03-04T12:13:07.718243+0000","last_became_active":"2024-03-04T12:13:06.732967+0000","last_became_peered":"2024-03-04T12:13:06.732967+0000","last_unstale":"2024-03-04T12:13:07.718243+0000","last_undegraded":"2024-03-04T12:13:07.718243+0000","last_fullsized":"2024-03-04T12:13:07.718243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:53:28.474198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731281+0000","last_change":"2024-03-04T12:13:05.736799+0000","last_active":"2024-03-04T12:13:07.731281+0000","last_peered":"2024-03-04T12:13:07.731281+0000","last_clean":"2024-03-04T12:13:07.731281+0000","last_became_active":"2024-03-04T12:13:05.736087+0000","last_became_peered":"2024-03-04T12:13:05.736087+0000","last_unstale":"2024-03-04T12:13:07.731281+0000","last_undegraded":"2024-03-04T12:13:07.731281+0000","last_fullsized":"2024-03-04T12:13:07.731281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:00:02.211146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.734922+0000","last_change":"2024-03-04T12:13:06.734268+0000","last_active":"2024-03-04T12:13:06.734922+0000","last_peered":"2024-03-04T12:13:06.734922+0000","last_clean":"2024-03-04T12:13:06.734922+0000","last_became_active":"2024-03-04T12:13:06.734016+0000","last_became_peered":"2024-03-04T12:13:06.734016+0000","last_unstale":"2024-03-04T12:13:06.734922+0000","last_undegraded":"2024-03-04T12:13:06.734922+0000","last_fullsized":"2024-03-04T12:13:06.734922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:14.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720136+0000","last_change":"2024-03-04T12:13:05.733324+0000","last_active":"2024-03-04T12:13:07.720136+0000","last_peered":"2024-03-04T12:13:07.720136+0000","last_clean":"2024-03-04T12:13:07.720136+0000","last_became_active":"2024-03-04T12:13:05.733025+0000","last_became_peered":"2024-03-04T12:13:05.733025+0000","last_unstale":"2024-03-04T12:13:07.720136+0000","last_undegraded":"2024-03-04T12:13:07.720136+0000","last_fullsized":"2024-03-04T12:13:07.720136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:56.514331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719429+0000","last_change":"2024-03-04T12:13:06.744828+0000","last_active":"2024-03-04T12:13:07.719429+0000","last_peered":"2024-03-04T12:13:07.719429+0000","last_clean":"2024-03-04T12:13:07.719429+0000","last_became_active":"2024-03-04T12:13:06.742465+0000","last_became_peered":"2024-03-04T12:13:06.742465+0000","last_unstale":"2024-03-04T12:13:07.719429+0000","last_undegraded":"2024-03-04T12:13:07.719429+0000","last_fullsized":"2024-03-04T12:13:07.719429+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.733163+0000","last_change":"2024-03-04T12:13:05.734842+0000","last_active":"2024-03-04T12:13:07.733163+0000","last_peered":"2024-03-04T12:13:07.733163+0000","last_clean":"2024-03-04T12:13:07.733163+0000","last_became_active":"2024-03-04T12:13:05.734531+0000","last_became_peered":"2024-03-04T12:13:05.734531+0000","last_unstale":"2024-03-04T12:13:07.733163+0000","last_undegraded":"2024-03-04T12:13:07.733163+0000","last_fullsized":"2024-03-04T12:13:07.733163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:51:49.288101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718228+0000","last_change":"2024-03-04T12:13:06.746153+0000","last_active":"2024-03-04T12:13:07.718228+0000","last_peered":"2024-03-04T12:13:07.718228+0000","last_clean":"2024-03-04T12:13:07.718228+0000","last_became_active":"2024-03-04T12:13:06.745897+0000","last_became_peered":"2024-03-04T12:13:06.745897+0000","last_unstale":"2024-03-04T12:13:07.718228+0000","last_undegraded":"2024-03-04T12:13:07.718228+0000","last_fullsized":"2024-03-04T12:13:07.718228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:25.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.721785+0000","last_change":"2024-03-04T12:13:05.737003+0000","last_active":"2024-03-04T12:13:06.721785+0000","last_peered":"2024-03-04T12:13:06.721785+0000","last_clean":"2024-03-04T12:13:06.721785+0000","last_became_active":"2024-03-04T12:13:05.736740+0000","last_became_peered":"2024-03-04T12:13:05.736740+0000","last_unstale":"2024-03-04T12:13:06.721785+0000","last_undegraded":"2024-03-04T12:13:06.721785+0000","last_fullsized":"2024-03-04T12:13:06.721785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:45.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.729158+0000","last_change":"2024-03-04T12:13:06.727366+0000","last_active":"2024-03-04T12:13:06.729158+0000","last_peered":"2024-03-04T12:13:06.729158+0000","last_clean":"2024-03-04T12:13:06.729158+0000","last_became_active":"2024-03-04T12:13:06.727120+0000","last_became_peered":"2024-03-04T12:13:06.727120+0000","last_unstale":"2024-03-04T12:13:06.729158+0000","last_undegraded":"2024-03-04T12:13:06.729158+0000","last_fullsized":"2024-03-04T12:13:06.729158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.726459+0000","last_change":"2024-03-04T12:13:05.732720+0000","last_active":"2024-03-04T12:13:06.726459+0000","last_peered":"2024-03-04T12:13:06.726459+0000","last_clean":"2024-03-04T12:13:06.726459+0000","last_became_active":"2024-03-04T12:13:05.732433+0000","last_became_peered":"2024-03-04T12:13:05.732433+0000","last_unstale":"2024-03-04T12:13:06.726459+0000","last_undegraded":"2024-03-04T12:13:06.726459+0000","last_fullsized":"2024-03-04T12:13:06.726459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:39.246078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.730822+0000","last_change":"2024-03-04T12:13:06.727385+0000","last_active":"2024-03-04T12:13:06.730822+0000","last_peered":"2024-03-04T12:13:06.730822+0000","last_clean":"2024-03-04T12:13:06.730822+0000","last_became_active":"2024-03-04T12:13:06.727076+0000","last_became_peered":"2024-03-04T12:13:06.727076+0000","last_unstale":"2024-03-04T12:13:06.730822+0000","last_undegraded":"2024-03-04T12:13:06.730822+0000","last_fullsized":"2024-03-04T12:13:06.730822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.729229+0000","last_change":"2024-03-04T12:13:05.733375+0000","last_active":"2024-03-04T12:13:06.729229+0000","last_peered":"2024-03-04T12:13:06.729229+0000","last_clean":"2024-03-04T12:13:06.729229+0000","last_became_active":"2024-03-04T12:13:05.733096+0000","last_became_peered":"2024-03-04T12:13:05.733096+0000","last_unstale":"2024-03-04T12:13:06.729229+0000","last_undegraded":"2024-03-04T12:13:06.729229+0000","last_fullsized":"2024-03-04T12:13:06.729229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:34:59.445308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745030+0000","last_change":"2024-03-04T12:13:06.740367+0000","last_active":"2024-03-04T12:13:06.745030+0000","last_peered":"2024-03-04T12:13:06.745030+0000","last_clean":"2024-03-04T12:13:06.745030+0000","last_became_active":"2024-03-04T12:13:06.739826+0000","last_became_peered":"2024-03-04T12:13:06.739826+0000","last_unstale":"2024-03-04T12:13:06.745030+0000","last_undegraded":"2024-03-04T12:13:06.745030+0000","last_fullsized":"2024-03-04T12:13:06.745030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.728203+0000","last_change":"2024-03-04T12:13:05.723793+0000","last_active":"2024-03-04T12:13:06.728203+0000","last_peered":"2024-03-04T12:13:06.728203+0000","last_clean":"2024-03-04T12:13:06.728203+0000","last_became_active":"2024-03-04T12:13:05.723496+0000","last_became_peered":"2024-03-04T12:13:05.723496+0000","last_unstale":"2024-03-04T12:13:06.728203+0000","last_undegraded":"2024-03-04T12:13:06.728203+0000","last_fullsized":"2024-03-04T12:13:06.728203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:54.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719270+0000","last_change":"2024-03-04T12:13:06.740765+0000","last_active":"2024-03-04T12:13:07.719270+0000","last_peered":"2024-03-04T12:13:07.719270+0000","last_clean":"2024-03-04T12:13:07.719270+0000","last_became_active":"2024-03-04T12:13:06.740498+0000","last_became_peered":"2024-03-04T12:13:06.740498+0000","last_unstale":"2024-03-04T12:13:07.719270+0000","last_undegraded":"2024-03-04T12:13:07.719270+0000","last_fullsized":"2024-03-04T12:13:07.719270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:51.258924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717867+0000","last_change":"2024-03-04T12:13:05.728827+0000","last_active":"2024-03-04T12:13:07.717867+0000","last_peered":"2024-03-04T12:13:07.717867+0000","last_clean":"2024-03-04T12:13:07.717867+0000","last_became_active":"2024-03-04T12:13:05.728560+0000","last_became_peered":"2024-03-04T12:13:05.728560+0000","last_unstale":"2024-03-04T12:13:07.717867+0000","last_undegraded":"2024-03-04T12:13:07.717867+0000","last_fullsized":"2024-03-04T12:13:07.717867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:45.391965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719823+0000","last_change":"2024-03-04T12:13:06.725267+0000","last_active":"2024-03-04T12:13:07.719823+0000","last_peered":"2024-03-04T12:13:07.719823+0000","last_clean":"2024-03-04T12:13:07.719823+0000","last_became_active":"2024-03-04T12:13:06.724965+0000","last_became_peered":"2024-03-04T12:13:06.724965+0000","last_unstale":"2024-03-04T12:13:07.719823+0000","last_undegraded":"2024-03-04T12:13:07.719823+0000","last_fullsized":"2024-03-04T12:13:07.719823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:45:48.052770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719299+0000","last_change":"2024-03-04T12:13:05.732899+0000","last_active":"2024-03-04T12:13:07.719299+0000","last_peered":"2024-03-04T12:13:07.719299+0000","last_clean":"2024-03-04T12:13:07.719299+0000","last_became_active":"2024-03-04T12:13:05.732563+0000","last_became_peered":"2024-03-04T12:13:05.732563+0000","last_unstale":"2024-03-04T12:13:07.719299+0000","last_undegraded":"2024-03-04T12:13:07.719299+0000","last_fullsized":"2024-03-04T12:13:07.719299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:13.739896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719459+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:13:07.719459+0000","last_peered":"2024-03-04T12:13:07.719459+0000","last_clean":"2024-03-04T12:13:07.719459+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:13:07.719459+0000","last_undegraded":"2024-03-04T12:13:07.719459+0000","last_fullsized":"2024-03-04T12:13:07.719459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":775386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:13:09.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T12:13:09.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.824+0000 7fda042ab640 1 Processor -- start 2024-03-04T12:13:09.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.824+0000 7fda042ab640 1 -- start start 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda042ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 0x7fd9fc15e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda042ab640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9fc0aeb20 0x7fd9fc071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda042ab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc15da40 con 0x7fd9fc071890 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda042ab640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15cbc0 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda042ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15ebb0 con 0x7fd9fc15d660 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01846640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9fc0aeb20 0x7fd9fc071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda02848640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9fc071890 0x7fd9fc150960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52214/0 (socket says 172.21.15.148:52214) 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda02848640 1 -- 172.21.15.148:0/967327015 learned_addr learned my addr 172.21.15.148:0/967327015 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda02047640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 0x7fd9fc15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01846640 1 -- 172.21.15.148:0/967327015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 msgr2=0x7fd9fc15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01846640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 0x7fd9fc15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01846640 1 -- 172.21.15.148:0/967327015 >> v1:172.21.15.148:6789/0 conn(0x7fd9fc071890 legacy=0x7fd9fc150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01846640 1 -- 172.21.15.148:0/967327015 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc0af090 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda01045640 1 -- 172.21.15.148:0/967327015 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503012013 0 0) 0x7fd9fc15da40 con 0x7fd9fc071890 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.825+0000 7fda02047640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 0x7fd9fc15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.826+0000 7fda01846640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc0aeb20 0x7fd9fc071350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df49f5ffe4469187 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.826+0000 7fda01045640 1 -- 172.21.15.148:0/967327015 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec038070 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.826+0000 7fda01045640 1 -- 172.21.15.148:0/967327015 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec031700 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.826+0000 7fda01045640 1 -- 172.21.15.148:0/967327015 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec031ad0 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 -- 172.21.15.148:0/967327015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc0aeb20 msgr2=0x7fd9fc071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc0aeb20 0x7fd9fc071350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097a0 tx=0x7fd9ec02b7a0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 -- 172.21.15.148:0/967327015 shutdown_connections 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc0aeb20 0x7fd9fc071350 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 --2- 172.21.15.148:0/967327015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc15d660 0x7fd9fc15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 -- 172.21.15.148:0/967327015 >> 172.21.15.148:0/967327015 conn(0x7fd9fc06acd0 msgr2=0x7fd9fc06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 -- 172.21.15.148:0/967327015 shutdown_connections 2024-03-04T12:13:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.827+0000 7fda042ab640 1 -- 172.21.15.148:0/967327015 wait complete. 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.828+0000 7fda042ab640 1 Processor -- start 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.828+0000 7fda042ab640 1 -- start start 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc071890 0x7fd9fc14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc0aeb20 0x7fd9fc137020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15afb0 con 0x7fd9fc15d660 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15acd0 con 0x7fd9fc071890 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda042ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15a660 con 0x7fd9fc0aeb20 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53662/0 (socket says 172.21.15.148:53662) 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 -- 172.21.15.148:0/30420295 learned_addr learned my addr 172.21.15.148:0/30420295 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda01846640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc0aeb20 0x7fd9fc137020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02047640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc071890 0x7fd9fc14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 -- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc0aeb20 msgr2=0x7fd9fc137020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc0aeb20 0x7fd9fc137020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 -- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc071890 msgr2=0x7fd9fc14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc071890 0x7fd9fc14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 -- 172.21.15.148:0/30420295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc145a20 con 0x7fd9fc15d660 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.829+0000 7fda02848640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c450 tx=0x7fd9f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.830+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018070 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.830+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4007500 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.830+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0cd150 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.830+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4004a20 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.830+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc0cd710 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.831+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc074f60 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.831+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd9f401b070 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.831+0000 7fd9f2ffd640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.831+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9f4089590 con 0x7fd9fc15d660 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.832+0000 7fda02047640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.832+0000 7fda02047640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 0x7fd9e006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f80097f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:09.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:09.836+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9fc145a20 con 0x7fd9fc15d660 2024-03-04T12:13:10.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.004+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 --> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9fc072460 con 0x7fd9e006b4d0 2024-03-04T12:13:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.010+0000 7fd9f2ffd640 1 -- 172.21.15.148:0/30420295 <== mgr.4104 v2:172.21.15.148:6832/2037319536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251926 (secure 0 0 0) 0x7fd9fc072460 con 0x7fd9e006b4d0 2024-03-04T12:13:10.014 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:10.015 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 msgr2=0x7fd9e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 0x7fd9e006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f80097f0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 msgr2=0x7fd9fc159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c450 tx=0x7fd9f400c910 comp rx=0 tx=0).stop 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 shutdown_connections 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fd9e006b4d0 0x7fd9e006d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9fc15d660 0x7fd9fc159190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc0aeb20 0x7fd9fc137020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 --2- 172.21.15.148:0/30420295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc071890 0x7fd9fc14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 >> 172.21.15.148:0/30420295 conn(0x7fd9fc06acd0 msgr2=0x7fd9fc0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 shutdown_connections 2024-03-04T12:13:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.013+0000 7fda042ab640 1 -- 172.21.15.148:0/30420295 wait complete. 2024-03-04T12:13:10.026 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-03-04T12:13:08.729134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220776,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753944,"statfs":{"total":773094113280,"available":772868038656,"internally_reserved":0,"allocated":4661248,"data_stored":2652244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019745"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718353+0000","last_change":"2024-03-04T12:13:06.737823+0000","last_active":"2024-03-04T12:13:07.718353+0000","last_peered":"2024-03-04T12:13:07.718353+0000","last_clean":"2024-03-04T12:13:07.718353+0000","last_became_active":"2024-03-04T12:13:06.737548+0000","last_became_peered":"2024-03-04T12:13:06.737548+0000","last_unstale":"2024-03-04T12:13:07.718353+0000","last_undegraded":"2024-03-04T12:13:07.718353+0000","last_fullsized":"2024-03-04T12:13:07.718353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:38:59.796983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718177+0000","last_change":"2024-03-04T12:13:06.725044+0000","last_active":"2024-03-04T12:13:07.718177+0000","last_peered":"2024-03-04T12:13:07.718177+0000","last_clean":"2024-03-04T12:13:07.718177+0000","last_became_active":"2024-03-04T12:13:06.724776+0000","last_became_peered":"2024-03-04T12:13:06.724776+0000","last_unstale":"2024-03-04T12:13:07.718177+0000","last_undegraded":"2024-03-04T12:13:07.718177+0000","last_fullsized":"2024-03-04T12:13:07.718177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:30.998287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.737454+0000","last_change":"2024-03-04T12:13:06.734974+0000","last_active":"2024-03-04T12:13:06.737454+0000","last_peered":"2024-03-04T12:13:06.737454+0000","last_clean":"2024-03-04T12:13:06.737454+0000","last_became_active":"2024-03-04T12:13:06.734718+0000","last_became_peered":"2024-03-04T12:13:06.734718+0000","last_unstale":"2024-03-04T12:13:06.737454+0000","last_undegraded":"2024-03-04T12:13:06.737454+0000","last_fullsized":"2024-03-04T12:13:06.737454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:57.771084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731496+0000","last_change":"2024-03-04T12:13:06.746238+0000","last_active":"2024-03-04T12:13:07.731496+0000","last_peered":"2024-03-04T12:13:07.731496+0000","last_clean":"2024-03-04T12:13:07.731496+0000","last_became_active":"2024-03-04T12:13:06.745975+0000","last_became_peered":"2024-03-04T12:13:06.745975+0000","last_unstale":"2024-03-04T12:13:07.731496+0000","last_undegraded":"2024-03-04T12:13:07.731496+0000","last_fullsized":"2024-03-04T12:13:07.731496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731486+0000","last_change":"2024-03-04T12:13:06.739596+0000","last_active":"2024-03-04T12:13:07.731486+0000","last_peered":"2024-03-04T12:13:07.731486+0000","last_clean":"2024-03-04T12:13:07.731486+0000","last_became_active":"2024-03-04T12:13:06.739355+0000","last_became_peered":"2024-03-04T12:13:06.739355+0000","last_unstale":"2024-03-04T12:13:07.731486+0000","last_undegraded":"2024-03-04T12:13:07.731486+0000","last_fullsized":"2024-03-04T12:13:07.731486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739806+0000","last_change":"2024-03-04T12:13:06.739253+0000","last_active":"2024-03-04T12:13:06.739806+0000","last_peered":"2024-03-04T12:13:06.739806+0000","last_clean":"2024-03-04T12:13:06.739806+0000","last_became_active":"2024-03-04T12:13:06.738665+0000","last_became_peered":"2024-03-04T12:13:06.738665+0000","last_unstale":"2024-03-04T12:13:06.739806+0000","last_undegraded":"2024-03-04T12:13:06.739806+0000","last_fullsized":"2024-03-04T12:13:06.739806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:57.771084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719666+0000","last_change":"2024-03-04T12:13:06.724647+0000","last_active":"2024-03-04T12:13:07.719666+0000","last_peered":"2024-03-04T12:13:07.719666+0000","last_clean":"2024-03-04T12:13:07.719666+0000","last_became_active":"2024-03-04T12:13:06.724371+0000","last_became_peered":"2024-03-04T12:13:06.724371+0000","last_unstale":"2024-03-04T12:13:07.719666+0000","last_undegraded":"2024-03-04T12:13:07.719666+0000","last_fullsized":"2024-03-04T12:13:07.719666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717888+0000","last_change":"2024-03-04T12:13:06.722774+0000","last_active":"2024-03-04T12:13:07.717888+0000","last_peered":"2024-03-04T12:13:07.717888+0000","last_clean":"2024-03-04T12:13:07.717888+0000","last_became_active":"2024-03-04T12:13:06.721953+0000","last_became_peered":"2024-03-04T12:13:06.721953+0000","last_unstale":"2024-03-04T12:13:07.717888+0000","last_undegraded":"2024-03-04T12:13:07.717888+0000","last_fullsized":"2024-03-04T12:13:07.717888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736803+0000","last_change":"2024-03-04T12:13:06.733531+0000","last_active":"2024-03-04T12:13:06.736803+0000","last_peered":"2024-03-04T12:13:06.736803+0000","last_clean":"2024-03-04T12:13:06.736803+0000","last_became_active":"2024-03-04T12:13:06.733285+0000","last_became_peered":"2024-03-04T12:13:06.733285+0000","last_unstale":"2024-03-04T12:13:06.736803+0000","last_undegraded":"2024-03-04T12:13:06.736803+0000","last_fullsized":"2024-03-04T12:13:06.736803+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:14.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719861+0000","last_change":"2024-03-04T12:13:06.731707+0000","last_active":"2024-03-04T12:13:07.719861+0000","last_peered":"2024-03-04T12:13:07.719861+0000","last_clean":"2024-03-04T12:13:07.719861+0000","last_became_active":"2024-03-04T12:13:06.730341+0000","last_became_peered":"2024-03-04T12:13:06.730341+0000","last_unstale":"2024-03-04T12:13:07.719861+0000","last_undegraded":"2024-03-04T12:13:07.719861+0000","last_fullsized":"2024-03-04T12:13:07.719861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:30.998287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720076+0000","last_change":"2024-03-04T12:13:06.743427+0000","last_active":"2024-03-04T12:13:07.720076+0000","last_peered":"2024-03-04T12:13:07.720076+0000","last_clean":"2024-03-04T12:13:07.720076+0000","last_became_active":"2024-03-04T12:13:06.743163+0000","last_became_peered":"2024-03-04T12:13:06.743163+0000","last_unstale":"2024-03-04T12:13:07.720076+0000","last_undegraded":"2024-03-04T12:13:07.720076+0000","last_fullsized":"2024-03-04T12:13:07.720076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:56:04.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.731452+0000","last_change":"2024-03-04T12:13:06.727332+0000","last_active":"2024-03-04T12:13:06.731452+0000","last_peered":"2024-03-04T12:13:06.731452+0000","last_clean":"2024-03-04T12:13:06.731452+0000","last_became_active":"2024-03-04T12:13:06.727060+0000","last_became_peered":"2024-03-04T12:13:06.727060+0000","last_unstale":"2024-03-04T12:13:06.731452+0000","last_undegraded":"2024-03-04T12:13:06.731452+0000","last_fullsized":"2024-03-04T12:13:06.731452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719801+0000","last_change":"2024-03-04T12:13:06.732492+0000","last_active":"2024-03-04T12:13:07.719801+0000","last_peered":"2024-03-04T12:13:07.719801+0000","last_clean":"2024-03-04T12:13:07.719801+0000","last_became_active":"2024-03-04T12:13:06.732206+0000","last_became_peered":"2024-03-04T12:13:06.732206+0000","last_unstale":"2024-03-04T12:13:07.719801+0000","last_undegraded":"2024-03-04T12:13:07.719801+0000","last_fullsized":"2024-03-04T12:13:07.719801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745289+0000","last_change":"2024-03-04T12:13:06.743131+0000","last_active":"2024-03-04T12:13:06.745289+0000","last_peered":"2024-03-04T12:13:06.745289+0000","last_clean":"2024-03-04T12:13:06.745289+0000","last_became_active":"2024-03-04T12:13:06.742852+0000","last_became_peered":"2024-03-04T12:13:06.742852+0000","last_unstale":"2024-03-04T12:13:06.745289+0000","last_undegraded":"2024-03-04T12:13:06.745289+0000","last_fullsized":"2024-03-04T12:13:06.745289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719334+0000","last_change":"2024-03-04T12:13:06.721302+0000","last_active":"2024-03-04T12:13:07.719334+0000","last_peered":"2024-03-04T12:13:07.719334+0000","last_clean":"2024-03-04T12:13:07.719334+0000","last_became_active":"2024-03-04T12:13:06.720979+0000","last_became_peered":"2024-03-04T12:13:06.720979+0000","last_unstale":"2024-03-04T12:13:07.719334+0000","last_undegraded":"2024-03-04T12:13:07.719334+0000","last_fullsized":"2024-03-04T12:13:07.719334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.960873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718689+0000","last_change":"2024-03-04T12:13:06.738208+0000","last_active":"2024-03-04T12:13:07.718689+0000","last_peered":"2024-03-04T12:13:07.718689+0000","last_clean":"2024-03-04T12:13:07.718689+0000","last_became_active":"2024-03-04T12:13:06.737922+0000","last_became_peered":"2024-03-04T12:13:06.737922+0000","last_unstale":"2024-03-04T12:13:07.718689+0000","last_undegraded":"2024-03-04T12:13:07.718689+0000","last_fullsized":"2024-03-04T12:13:07.718689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:16:56.601953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719089+0000","last_change":"2024-03-04T12:13:06.733779+0000","last_active":"2024-03-04T12:13:07.719089+0000","last_peered":"2024-03-04T12:13:07.719089+0000","last_clean":"2024-03-04T12:13:07.719089+0000","last_became_active":"2024-03-04T12:13:06.733443+0000","last_became_peered":"2024-03-04T12:13:06.733443+0000","last_unstale":"2024-03-04T12:13:07.719089+0000","last_undegraded":"2024-03-04T12:13:07.719089+0000","last_fullsized":"2024-03-04T12:13:07.719089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:39:53.755220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749340+0000","last_change":"2024-03-04T12:13:06.746884+0000","last_active":"2024-03-04T12:13:06.749340+0000","last_peered":"2024-03-04T12:13:06.749340+0000","last_clean":"2024-03-04T12:13:06.749340+0000","last_became_active":"2024-03-04T12:13:06.746607+0000","last_became_peered":"2024-03-04T12:13:06.746607+0000","last_unstale":"2024-03-04T12:13:06.749340+0000","last_undegraded":"2024-03-04T12:13:06.749340+0000","last_fullsized":"2024-03-04T12:13:06.749340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731552+0000","last_change":"2024-03-04T12:13:06.735464+0000","last_active":"2024-03-04T12:13:07.731552+0000","last_peered":"2024-03-04T12:13:07.731552+0000","last_clean":"2024-03-04T12:13:07.731552+0000","last_became_active":"2024-03-04T12:13:06.735175+0000","last_became_peered":"2024-03-04T12:13:06.735175+0000","last_unstale":"2024-03-04T12:13:07.731552+0000","last_undegraded":"2024-03-04T12:13:07.731552+0000","last_fullsized":"2024-03-04T12:13:07.731552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747198+0000","last_change":"2024-03-04T12:13:06.746223+0000","last_active":"2024-03-04T12:13:06.747198+0000","last_peered":"2024-03-04T12:13:06.747198+0000","last_clean":"2024-03-04T12:13:06.747198+0000","last_became_active":"2024-03-04T12:13:06.745931+0000","last_became_peered":"2024-03-04T12:13:06.745931+0000","last_unstale":"2024-03-04T12:13:06.747198+0000","last_undegraded":"2024-03-04T12:13:06.747198+0000","last_fullsized":"2024-03-04T12:13:06.747198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719782+0000","last_change":"2024-03-04T12:13:06.730962+0000","last_active":"2024-03-04T12:13:07.719782+0000","last_peered":"2024-03-04T12:13:07.719782+0000","last_clean":"2024-03-04T12:13:07.719782+0000","last_became_active":"2024-03-04T12:13:06.730502+0000","last_became_peered":"2024-03-04T12:13:06.730502+0000","last_unstale":"2024-03-04T12:13:07.719782+0000","last_undegraded":"2024-03-04T12:13:07.719782+0000","last_fullsized":"2024-03-04T12:13:07.719782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747351+0000","last_change":"2024-03-04T12:13:06.745158+0000","last_active":"2024-03-04T12:13:06.747351+0000","last_peered":"2024-03-04T12:13:06.747351+0000","last_clean":"2024-03-04T12:13:06.747351+0000","last_became_active":"2024-03-04T12:13:06.744818+0000","last_became_peered":"2024-03-04T12:13:06.744818+0000","last_unstale":"2024-03-04T12:13:06.747351+0000","last_undegraded":"2024-03-04T12:13:06.747351+0000","last_fullsized":"2024-03-04T12:13:06.747351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719116+0000","last_change":"2024-03-04T12:13:06.739952+0000","last_active":"2024-03-04T12:13:07.719116+0000","last_peered":"2024-03-04T12:13:07.719116+0000","last_clean":"2024-03-04T12:13:07.719116+0000","last_became_active":"2024-03-04T12:13:06.739672+0000","last_became_peered":"2024-03-04T12:13:06.739672+0000","last_unstale":"2024-03-04T12:13:07.719116+0000","last_undegraded":"2024-03-04T12:13:07.719116+0000","last_fullsized":"2024-03-04T12:13:07.719116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:17.842475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731486+0000","last_change":"2024-03-04T12:13:06.731651+0000","last_active":"2024-03-04T12:13:07.731486+0000","last_peered":"2024-03-04T12:13:07.731486+0000","last_clean":"2024-03-04T12:13:07.731486+0000","last_became_active":"2024-03-04T12:13:06.731383+0000","last_became_peered":"2024-03-04T12:13:06.731383+0000","last_unstale":"2024-03-04T12:13:07.731486+0000","last_undegraded":"2024-03-04T12:13:07.731486+0000","last_fullsized":"2024-03-04T12:13:07.731486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739484+0000","last_change":"2024-03-04T12:13:06.738594+0000","last_active":"2024-03-04T12:13:06.739484+0000","last_peered":"2024-03-04T12:13:06.739484+0000","last_clean":"2024-03-04T12:13:06.739484+0000","last_became_active":"2024-03-04T12:13:06.738329+0000","last_became_peered":"2024-03-04T12:13:06.738329+0000","last_unstale":"2024-03-04T12:13:06.739484+0000","last_undegraded":"2024-03-04T12:13:06.739484+0000","last_fullsized":"2024-03-04T12:13:06.739484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:46.130721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.734868+0000","last_change":"2024-03-04T12:13:06.733981+0000","last_active":"2024-03-04T12:13:06.734868+0000","last_peered":"2024-03-04T12:13:06.734868+0000","last_clean":"2024-03-04T12:13:06.734868+0000","last_became_active":"2024-03-04T12:13:06.733680+0000","last_became_peered":"2024-03-04T12:13:06.733680+0000","last_unstale":"2024-03-04T12:13:06.734868+0000","last_undegraded":"2024-03-04T12:13:06.734868+0000","last_fullsized":"2024-03-04T12:13:06.734868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739221+0000","last_change":"2024-03-04T12:13:06.733781+0000","last_active":"2024-03-04T12:13:06.739221+0000","last_peered":"2024-03-04T12:13:06.739221+0000","last_clean":"2024-03-04T12:13:06.739221+0000","last_became_active":"2024-03-04T12:13:06.733527+0000","last_became_peered":"2024-03-04T12:13:06.733527+0000","last_unstale":"2024-03-04T12:13:06.739221+0000","last_undegraded":"2024-03-04T12:13:06.739221+0000","last_fullsized":"2024-03-04T12:13:06.739221+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:55.200883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720083+0000","last_change":"2024-03-04T12:13:06.720879+0000","last_active":"2024-03-04T12:13:07.720083+0000","last_peered":"2024-03-04T12:13:07.720083+0000","last_clean":"2024-03-04T12:13:07.720083+0000","last_became_active":"2024-03-04T12:13:06.720605+0000","last_became_peered":"2024-03-04T12:13:06.720605+0000","last_unstale":"2024-03-04T12:13:07.720083+0000","last_undegraded":"2024-03-04T12:13:07.720083+0000","last_fullsized":"2024-03-04T12:13:07.720083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:53:28.474198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739283+0000","last_change":"2024-03-04T12:13:06.733427+0000","last_active":"2024-03-04T12:13:06.739283+0000","last_peered":"2024-03-04T12:13:06.739283+0000","last_clean":"2024-03-04T12:13:06.739283+0000","last_became_active":"2024-03-04T12:13:06.733122+0000","last_became_peered":"2024-03-04T12:13:06.733122+0000","last_unstale":"2024-03-04T12:13:06.739283+0000","last_undegraded":"2024-03-04T12:13:06.739283+0000","last_fullsized":"2024-03-04T12:13:06.739283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:57.298861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.739346+0000","last_change":"2024-03-04T12:13:06.737090+0000","last_active":"2024-03-04T12:13:06.739346+0000","last_peered":"2024-03-04T12:13:06.739346+0000","last_clean":"2024-03-04T12:13:06.739346+0000","last_became_active":"2024-03-04T12:13:06.736832+0000","last_became_peered":"2024-03-04T12:13:06.736832+0000","last_unstale":"2024-03-04T12:13:06.739346+0000","last_undegraded":"2024-03-04T12:13:06.739346+0000","last_fullsized":"2024-03-04T12:13:06.739346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:07:37.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719208+0000","last_change":"2024-03-04T12:13:06.735954+0000","last_active":"2024-03-04T12:13:07.719208+0000","last_peered":"2024-03-04T12:13:07.719208+0000","last_clean":"2024-03-04T12:13:07.719208+0000","last_became_active":"2024-03-04T12:13:06.735672+0000","last_became_peered":"2024-03-04T12:13:06.735672+0000","last_unstale":"2024-03-04T12:13:07.719208+0000","last_undegraded":"2024-03-04T12:13:07.719208+0000","last_fullsized":"2024-03-04T12:13:07.719208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.960873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.731552+0000","last_change":"2024-03-04T12:13:06.730175+0000","last_active":"2024-03-04T12:13:06.731552+0000","last_peered":"2024-03-04T12:13:06.731552+0000","last_clean":"2024-03-04T12:13:06.731552+0000","last_became_active":"2024-03-04T12:13:06.729927+0000","last_became_peered":"2024-03-04T12:13:06.729927+0000","last_unstale":"2024-03-04T12:13:06.731552+0000","last_undegraded":"2024-03-04T12:13:06.731552+0000","last_fullsized":"2024-03-04T12:13:06.731552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719373+0000","last_change":"2024-03-04T12:13:06.735462+0000","last_active":"2024-03-04T12:13:07.719373+0000","last_peered":"2024-03-04T12:13:07.719373+0000","last_clean":"2024-03-04T12:13:07.719373+0000","last_became_active":"2024-03-04T12:13:06.734980+0000","last_became_peered":"2024-03-04T12:13:06.734980+0000","last_unstale":"2024-03-04T12:13:07.719373+0000","last_undegraded":"2024-03-04T12:13:07.719373+0000","last_fullsized":"2024-03-04T12:13:07.719373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:12:58.923163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747940+0000","last_change":"2024-03-04T12:13:05.736181+0000","last_active":"2024-03-04T12:13:06.747940+0000","last_peered":"2024-03-04T12:13:06.747940+0000","last_clean":"2024-03-04T12:13:06.747940+0000","last_became_active":"2024-03-04T12:13:05.735906+0000","last_became_peered":"2024-03-04T12:13:05.735906+0000","last_unstale":"2024-03-04T12:13:06.747940+0000","last_undegraded":"2024-03-04T12:13:06.747940+0000","last_fullsized":"2024-03-04T12:13:06.747940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:52:54.955194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.743516+0000","last_change":"2024-03-04T12:13:06.742364+0000","last_active":"2024-03-04T12:13:06.743516+0000","last_peered":"2024-03-04T12:13:06.743516+0000","last_clean":"2024-03-04T12:13:06.743516+0000","last_became_active":"2024-03-04T12:13:06.742108+0000","last_became_peered":"2024-03-04T12:13:06.742108+0000","last_unstale":"2024-03-04T12:13:06.743516+0000","last_undegraded":"2024-03-04T12:13:06.743516+0000","last_fullsized":"2024-03-04T12:13:06.743516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:57.516181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749320+0000","last_change":"2024-03-04T12:13:05.735007+0000","last_active":"2024-03-04T12:13:06.749320+0000","last_peered":"2024-03-04T12:13:06.749320+0000","last_clean":"2024-03-04T12:13:06.749320+0000","last_became_active":"2024-03-04T12:13:05.734294+0000","last_became_peered":"2024-03-04T12:13:05.734294+0000","last_unstale":"2024-03-04T12:13:06.749320+0000","last_undegraded":"2024-03-04T12:13:06.749320+0000","last_fullsized":"2024-03-04T12:13:06.749320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:38.435861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.742621+0000","last_change":"2024-03-04T12:13:06.739556+0000","last_active":"2024-03-04T12:13:06.742621+0000","last_peered":"2024-03-04T12:13:06.742621+0000","last_clean":"2024-03-04T12:13:06.742621+0000","last_became_active":"2024-03-04T12:13:06.739113+0000","last_became_peered":"2024-03-04T12:13:06.739113+0000","last_unstale":"2024-03-04T12:13:06.742621+0000","last_undegraded":"2024-03-04T12:13:06.742621+0000","last_fullsized":"2024-03-04T12:13:06.742621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:46.130721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.742858+0000","last_change":"2024-03-04T12:13:05.736255+0000","last_active":"2024-03-04T12:13:06.742858+0000","last_peered":"2024-03-04T12:13:06.742858+0000","last_clean":"2024-03-04T12:13:06.742858+0000","last_became_active":"2024-03-04T12:13:05.735991+0000","last_became_peered":"2024-03-04T12:13:05.735991+0000","last_unstale":"2024-03-04T12:13:06.742858+0000","last_undegraded":"2024-03-04T12:13:06.742858+0000","last_fullsized":"2024-03-04T12:13:06.742858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:39.246078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718337+0000","last_change":"2024-03-04T12:13:06.735687+0000","last_active":"2024-03-04T12:13:07.718337+0000","last_peered":"2024-03-04T12:13:07.718337+0000","last_clean":"2024-03-04T12:13:07.718337+0000","last_became_active":"2024-03-04T12:13:06.735409+0000","last_became_peered":"2024-03-04T12:13:06.735409+0000","last_unstale":"2024-03-04T12:13:07.718337+0000","last_undegraded":"2024-03-04T12:13:07.718337+0000","last_fullsized":"2024-03-04T12:13:07.718337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:00.391256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745284+0000","last_change":"2024-03-04T12:13:05.726597+0000","last_active":"2024-03-04T12:13:06.745284+0000","last_peered":"2024-03-04T12:13:06.745284+0000","last_clean":"2024-03-04T12:13:06.745284+0000","last_became_active":"2024-03-04T12:13:05.726290+0000","last_became_peered":"2024-03-04T12:13:05.726290+0000","last_unstale":"2024-03-04T12:13:06.745284+0000","last_undegraded":"2024-03-04T12:13:06.745284+0000","last_fullsized":"2024-03-04T12:13:06.745284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:07:36.898994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736165+0000","last_change":"2024-03-04T12:13:06.730947+0000","last_active":"2024-03-04T12:13:06.736165+0000","last_peered":"2024-03-04T12:13:06.736165+0000","last_clean":"2024-03-04T12:13:06.736165+0000","last_became_active":"2024-03-04T12:13:06.730684+0000","last_became_peered":"2024-03-04T12:13:06.730684+0000","last_unstale":"2024-03-04T12:13:06.736165+0000","last_undegraded":"2024-03-04T12:13:06.736165+0000","last_fullsized":"2024-03-04T12:13:06.736165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719936+0000","last_change":"2024-03-04T12:13:05.736036+0000","last_active":"2024-03-04T12:13:07.719936+0000","last_peered":"2024-03-04T12:13:07.719936+0000","last_clean":"2024-03-04T12:13:07.719936+0000","last_became_active":"2024-03-04T12:13:05.735708+0000","last_became_peered":"2024-03-04T12:13:05.735708+0000","last_unstale":"2024-03-04T12:13:07.719936+0000","last_undegraded":"2024-03-04T12:13:07.719936+0000","last_fullsized":"2024-03-04T12:13:07.719936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:44.902387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719526+0000","last_change":"2024-03-04T12:13:06.725433+0000","last_active":"2024-03-04T12:13:07.719526+0000","last_peered":"2024-03-04T12:13:07.719526+0000","last_clean":"2024-03-04T12:13:07.719526+0000","last_became_active":"2024-03-04T12:13:06.725144+0000","last_became_peered":"2024-03-04T12:13:06.725144+0000","last_unstale":"2024-03-04T12:13:07.719526+0000","last_undegraded":"2024-03-04T12:13:07.719526+0000","last_fullsized":"2024-03-04T12:13:07.719526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:51.258924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719081+0000","last_change":"2024-03-04T12:13:05.724819+0000","last_active":"2024-03-04T12:13:07.719081+0000","last_peered":"2024-03-04T12:13:07.719081+0000","last_clean":"2024-03-04T12:13:07.719081+0000","last_became_active":"2024-03-04T12:13:05.724561+0000","last_became_peered":"2024-03-04T12:13:05.724561+0000","last_unstale":"2024-03-04T12:13:07.719081+0000","last_undegraded":"2024-03-04T12:13:07.719081+0000","last_fullsized":"2024-03-04T12:13:07.719081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:44.902387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745010+0000","last_change":"2024-03-04T12:13:06.740150+0000","last_active":"2024-03-04T12:13:06.745010+0000","last_peered":"2024-03-04T12:13:06.745010+0000","last_clean":"2024-03-04T12:13:06.745010+0000","last_became_active":"2024-03-04T12:13:06.739502+0000","last_became_peered":"2024-03-04T12:13:06.739502+0000","last_unstale":"2024-03-04T12:13:06.745010+0000","last_undegraded":"2024-03-04T12:13:06.745010+0000","last_fullsized":"2024-03-04T12:13:06.745010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:39.437711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725514+0000","last_change":"2024-03-04T12:13:05.733273+0000","last_active":"2024-03-04T12:13:06.725514+0000","last_peered":"2024-03-04T12:13:06.725514+0000","last_clean":"2024-03-04T12:13:06.725514+0000","last_became_active":"2024-03-04T12:13:05.733005+0000","last_became_peered":"2024-03-04T12:13:05.733005+0000","last_unstale":"2024-03-04T12:13:06.725514+0000","last_undegraded":"2024-03-04T12:13:06.725514+0000","last_fullsized":"2024-03-04T12:13:06.725514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:45.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718171+0000","last_change":"2024-03-04T12:13:06.732782+0000","last_active":"2024-03-04T12:13:07.718171+0000","last_peered":"2024-03-04T12:13:07.718171+0000","last_clean":"2024-03-04T12:13:07.718171+0000","last_became_active":"2024-03-04T12:13:06.732451+0000","last_became_peered":"2024-03-04T12:13:06.732451+0000","last_unstale":"2024-03-04T12:13:07.718171+0000","last_undegraded":"2024-03-04T12:13:07.718171+0000","last_fullsized":"2024-03-04T12:13:07.718171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:45:48.052770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725723+0000","last_change":"2024-03-04T12:13:05.731307+0000","last_active":"2024-03-04T12:13:06.725723+0000","last_peered":"2024-03-04T12:13:06.725723+0000","last_clean":"2024-03-04T12:13:06.725723+0000","last_became_active":"2024-03-04T12:13:05.731030+0000","last_became_peered":"2024-03-04T12:13:05.731030+0000","last_unstale":"2024-03-04T12:13:06.725723+0000","last_undegraded":"2024-03-04T12:13:06.725723+0000","last_fullsized":"2024-03-04T12:13:06.725723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:56.297011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731503+0000","last_change":"2024-03-04T12:13:06.746689+0000","last_active":"2024-03-04T12:13:07.731503+0000","last_peered":"2024-03-04T12:13:07.731503+0000","last_clean":"2024-03-04T12:13:07.731503+0000","last_became_active":"2024-03-04T12:13:06.746409+0000","last_became_peered":"2024-03-04T12:13:06.746409+0000","last_unstale":"2024-03-04T12:13:07.731503+0000","last_undegraded":"2024-03-04T12:13:07.731503+0000","last_fullsized":"2024-03-04T12:13:07.731503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718424+0000","last_change":"2024-03-04T12:13:05.728812+0000","last_active":"2024-03-04T12:13:07.718424+0000","last_peered":"2024-03-04T12:13:07.718424+0000","last_clean":"2024-03-04T12:13:07.718424+0000","last_became_active":"2024-03-04T12:13:05.728535+0000","last_became_peered":"2024-03-04T12:13:05.728535+0000","last_unstale":"2024-03-04T12:13:07.718424+0000","last_undegraded":"2024-03-04T12:13:07.718424+0000","last_fullsized":"2024-03-04T12:13:07.718424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:21:49.594403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719469+0000","last_change":"2024-03-04T12:13:06.738097+0000","last_active":"2024-03-04T12:13:07.719469+0000","last_peered":"2024-03-04T12:13:07.719469+0000","last_clean":"2024-03-04T12:13:07.719469+0000","last_became_active":"2024-03-04T12:13:06.737820+0000","last_became_peered":"2024-03-04T12:13:06.737820+0000","last_unstale":"2024-03-04T12:13:07.719469+0000","last_undegraded":"2024-03-04T12:13:07.719469+0000","last_fullsized":"2024-03-04T12:13:07.719469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719995+0000","last_change":"2024-03-04T12:13:05.730546+0000","last_active":"2024-03-04T12:13:07.719995+0000","last_peered":"2024-03-04T12:13:07.719995+0000","last_clean":"2024-03-04T12:13:07.719995+0000","last_became_active":"2024-03-04T12:13:05.729638+0000","last_became_peered":"2024-03-04T12:13:05.729638+0000","last_unstale":"2024-03-04T12:13:07.719995+0000","last_undegraded":"2024-03-04T12:13:07.719995+0000","last_fullsized":"2024-03-04T12:13:07.719995+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:54:56.769234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.738230+0000","last_change":"2024-03-04T12:13:06.737086+0000","last_active":"2024-03-04T12:13:06.738230+0000","last_peered":"2024-03-04T12:13:06.738230+0000","last_clean":"2024-03-04T12:13:06.738230+0000","last_became_active":"2024-03-04T12:13:06.736813+0000","last_became_peered":"2024-03-04T12:13:06.736813+0000","last_unstale":"2024-03-04T12:13:06.738230+0000","last_undegraded":"2024-03-04T12:13:06.738230+0000","last_fullsized":"2024-03-04T12:13:06.738230+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:46.393815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720140+0000","last_change":"2024-03-04T12:13:05.731299+0000","last_active":"2024-03-04T12:13:07.720140+0000","last_peered":"2024-03-04T12:13:07.720140+0000","last_clean":"2024-03-04T12:13:07.720140+0000","last_became_active":"2024-03-04T12:13:05.730996+0000","last_became_peered":"2024-03-04T12:13:05.730996+0000","last_unstale":"2024-03-04T12:13:07.720140+0000","last_undegraded":"2024-03-04T12:13:07.720140+0000","last_fullsized":"2024-03-04T12:13:07.720140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:13.739896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.747305+0000","last_change":"2024-03-04T12:13:06.744627+0000","last_active":"2024-03-04T12:13:06.747305+0000","last_peered":"2024-03-04T12:13:06.747305+0000","last_clean":"2024-03-04T12:13:06.747305+0000","last_became_active":"2024-03-04T12:13:06.744368+0000","last_became_peered":"2024-03-04T12:13:06.744368+0000","last_unstale":"2024-03-04T12:13:06.747305+0000","last_undegraded":"2024-03-04T12:13:06.747305+0000","last_fullsized":"2024-03-04T12:13:06.747305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:30.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718299+0000","last_change":"2024-03-04T12:13:05.725302+0000","last_active":"2024-03-04T12:13:07.718299+0000","last_peered":"2024-03-04T12:13:07.718299+0000","last_clean":"2024-03-04T12:13:07.718299+0000","last_became_active":"2024-03-04T12:13:05.725040+0000","last_became_peered":"2024-03-04T12:13:05.725040+0000","last_unstale":"2024-03-04T12:13:07.718299+0000","last_undegraded":"2024-03-04T12:13:07.718299+0000","last_fullsized":"2024-03-04T12:13:07.718299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:40:46.456139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736827+0000","last_change":"2024-03-04T12:13:06.733377+0000","last_active":"2024-03-04T12:13:06.736827+0000","last_peered":"2024-03-04T12:13:06.736827+0000","last_clean":"2024-03-04T12:13:06.736827+0000","last_became_active":"2024-03-04T12:13:06.733101+0000","last_became_peered":"2024-03-04T12:13:06.733101+0000","last_unstale":"2024-03-04T12:13:06.736827+0000","last_undegraded":"2024-03-04T12:13:06.736827+0000","last_fullsized":"2024-03-04T12:13:06.736827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718298+0000","last_change":"2024-03-04T12:13:05.734422+0000","last_active":"2024-03-04T12:13:07.718298+0000","last_peered":"2024-03-04T12:13:07.718298+0000","last_clean":"2024-03-04T12:13:07.718298+0000","last_became_active":"2024-03-04T12:13:05.734160+0000","last_became_peered":"2024-03-04T12:13:05.734160+0000","last_unstale":"2024-03-04T12:13:07.718298+0000","last_undegraded":"2024-03-04T12:13:07.718298+0000","last_fullsized":"2024-03-04T12:13:07.718298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:54.000013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717873+0000","last_change":"2024-03-04T12:13:06.730358+0000","last_active":"2024-03-04T12:13:07.717873+0000","last_peered":"2024-03-04T12:13:07.717873+0000","last_clean":"2024-03-04T12:13:07.717873+0000","last_became_active":"2024-03-04T12:13:06.730109+0000","last_became_peered":"2024-03-04T12:13:06.730109+0000","last_unstale":"2024-03-04T12:13:07.717873+0000","last_undegraded":"2024-03-04T12:13:07.717873+0000","last_fullsized":"2024-03-04T12:13:07.717873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.725660+0000","last_change":"2024-03-04T12:13:05.732806+0000","last_active":"2024-03-04T12:13:06.725660+0000","last_peered":"2024-03-04T12:13:06.725660+0000","last_clean":"2024-03-04T12:13:06.725660+0000","last_became_active":"2024-03-04T12:13:05.732567+0000","last_became_peered":"2024-03-04T12:13:05.732567+0000","last_unstale":"2024-03-04T12:13:06.725660+0000","last_undegraded":"2024-03-04T12:13:06.725660+0000","last_fullsized":"2024-03-04T12:13:06.725660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:52:38.435861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718832+0000","last_change":"2024-03-04T12:13:06.736969+0000","last_active":"2024-03-04T12:13:07.718832+0000","last_peered":"2024-03-04T12:13:07.718832+0000","last_clean":"2024-03-04T12:13:07.718832+0000","last_became_active":"2024-03-04T12:13:06.736682+0000","last_became_peered":"2024-03-04T12:13:06.736682+0000","last_unstale":"2024-03-04T12:13:07.718832+0000","last_undegraded":"2024-03-04T12:13:07.718832+0000","last_fullsized":"2024-03-04T12:13:07.718832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:39:11.380195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.724993+0000","last_change":"2024-03-04T12:13:05.738768+0000","last_active":"2024-03-04T12:13:06.724993+0000","last_peered":"2024-03-04T12:13:06.724993+0000","last_clean":"2024-03-04T12:13:06.724993+0000","last_became_active":"2024-03-04T12:13:05.738446+0000","last_became_peered":"2024-03-04T12:13:05.738446+0000","last_unstale":"2024-03-04T12:13:06.724993+0000","last_undegraded":"2024-03-04T12:13:06.724993+0000","last_fullsized":"2024-03-04T12:13:06.724993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:01:05.209750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718094+0000","last_change":"2024-03-04T12:13:06.729755+0000","last_active":"2024-03-04T12:13:07.718094+0000","last_peered":"2024-03-04T12:13:07.718094+0000","last_clean":"2024-03-04T12:13:07.718094+0000","last_became_active":"2024-03-04T12:13:06.729490+0000","last_became_peered":"2024-03-04T12:13:06.729490+0000","last_unstale":"2024-03-04T12:13:07.718094+0000","last_undegraded":"2024-03-04T12:13:07.718094+0000","last_fullsized":"2024-03-04T12:13:07.718094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:34:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718810+0000","last_change":"2024-03-04T12:13:05.724337+0000","last_active":"2024-03-04T12:13:07.718810+0000","last_peered":"2024-03-04T12:13:07.718810+0000","last_clean":"2024-03-04T12:13:07.718810+0000","last_became_active":"2024-03-04T12:13:05.724047+0000","last_became_peered":"2024-03-04T12:13:05.724047+0000","last_unstale":"2024-03-04T12:13:07.718810+0000","last_undegraded":"2024-03-04T12:13:07.718810+0000","last_fullsized":"2024-03-04T12:13:07.718810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:24.459143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718309+0000","last_change":"2024-03-04T12:13:06.736490+0000","last_active":"2024-03-04T12:13:07.718309+0000","last_peered":"2024-03-04T12:13:07.718309+0000","last_clean":"2024-03-04T12:13:07.718309+0000","last_became_active":"2024-03-04T12:13:06.735977+0000","last_became_peered":"2024-03-04T12:13:06.735977+0000","last_unstale":"2024-03-04T12:13:07.718309+0000","last_undegraded":"2024-03-04T12:13:07.718309+0000","last_fullsized":"2024-03-04T12:13:07.718309+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:56:04.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720024+0000","last_change":"2024-03-04T12:13:05.736454+0000","last_active":"2024-03-04T12:13:07.720024+0000","last_peered":"2024-03-04T12:13:07.720024+0000","last_clean":"2024-03-04T12:13:07.720024+0000","last_became_active":"2024-03-04T12:13:05.736151+0000","last_became_peered":"2024-03-04T12:13:05.736151+0000","last_unstale":"2024-03-04T12:13:07.720024+0000","last_undegraded":"2024-03-04T12:13:07.720024+0000","last_fullsized":"2024-03-04T12:13:07.720024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:45.391965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719989+0000","last_change":"2024-03-04T12:13:06.731886+0000","last_active":"2024-03-04T12:13:07.719989+0000","last_peered":"2024-03-04T12:13:07.719989+0000","last_clean":"2024-03-04T12:13:07.719989+0000","last_became_active":"2024-03-04T12:13:06.731577+0000","last_became_peered":"2024-03-04T12:13:06.731577+0000","last_unstale":"2024-03-04T12:13:07.719989+0000","last_undegraded":"2024-03-04T12:13:07.719989+0000","last_fullsized":"2024-03-04T12:13:07.719989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:25.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.749826+0000","last_change":"2024-03-04T12:13:05.732482+0000","last_active":"2024-03-04T12:13:06.749826+0000","last_peered":"2024-03-04T12:13:06.749826+0000","last_clean":"2024-03-04T12:13:06.749826+0000","last_became_active":"2024-03-04T12:13:05.732195+0000","last_became_peered":"2024-03-04T12:13:05.732195+0000","last_unstale":"2024-03-04T12:13:06.749826+0000","last_undegraded":"2024-03-04T12:13:06.749826+0000","last_fullsized":"2024-03-04T12:13:06.749826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:34:59.445308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.743494+0000","last_change":"2024-03-04T12:13:06.740765+0000","last_active":"2024-03-04T12:13:06.743494+0000","last_peered":"2024-03-04T12:13:06.743494+0000","last_clean":"2024-03-04T12:13:06.743494+0000","last_became_active":"2024-03-04T12:13:06.740498+0000","last_became_peered":"2024-03-04T12:13:06.740498+0000","last_unstale":"2024-03-04T12:13:06.743494+0000","last_undegraded":"2024-03-04T12:13:06.743494+0000","last_fullsized":"2024-03-04T12:13:06.743494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731597+0000","last_change":"2024-03-04T12:13:05.731193+0000","last_active":"2024-03-04T12:13:07.731597+0000","last_peered":"2024-03-04T12:13:07.731597+0000","last_clean":"2024-03-04T12:13:07.731597+0000","last_became_active":"2024-03-04T12:13:05.730883+0000","last_became_peered":"2024-03-04T12:13:05.730883+0000","last_unstale":"2024-03-04T12:13:07.731597+0000","last_undegraded":"2024-03-04T12:13:07.731597+0000","last_fullsized":"2024-03-04T12:13:07.731597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:01:05.209750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731698+0000","last_change":"2024-03-04T12:13:06.724271+0000","last_active":"2024-03-04T12:13:07.731698+0000","last_peered":"2024-03-04T12:13:07.731698+0000","last_clean":"2024-03-04T12:13:07.731698+0000","last_became_active":"2024-03-04T12:13:06.723942+0000","last_became_peered":"2024-03-04T12:13:06.723942+0000","last_unstale":"2024-03-04T12:13:07.731698+0000","last_undegraded":"2024-03-04T12:13:07.731698+0000","last_fullsized":"2024-03-04T12:13:07.731698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:45.904237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.728594+0000","last_change":"2024-03-04T12:13:05.736760+0000","last_active":"2024-03-04T12:13:06.728594+0000","last_peered":"2024-03-04T12:13:06.728594+0000","last_clean":"2024-03-04T12:13:06.728594+0000","last_became_active":"2024-03-04T12:13:05.736432+0000","last_became_peered":"2024-03-04T12:13:05.736432+0000","last_unstale":"2024-03-04T12:13:06.728594+0000","last_undegraded":"2024-03-04T12:13:06.728594+0000","last_fullsized":"2024-03-04T12:13:06.728594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:24.459143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719412+0000","last_change":"2024-03-04T12:13:06.731814+0000","last_active":"2024-03-04T12:13:07.719412+0000","last_peered":"2024-03-04T12:13:07.719412+0000","last_clean":"2024-03-04T12:13:07.719412+0000","last_became_active":"2024-03-04T12:13:06.731540+0000","last_became_peered":"2024-03-04T12:13:06.731540+0000","last_unstale":"2024-03-04T12:13:07.719412+0000","last_undegraded":"2024-03-04T12:13:07.719412+0000","last_fullsized":"2024-03-04T12:13:07.719412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:05:56.463181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731515+0000","last_change":"2024-03-04T12:13:05.736704+0000","last_active":"2024-03-04T12:13:07.731515+0000","last_peered":"2024-03-04T12:13:07.731515+0000","last_clean":"2024-03-04T12:13:07.731515+0000","last_became_active":"2024-03-04T12:13:05.735570+0000","last_became_peered":"2024-03-04T12:13:05.735570+0000","last_unstale":"2024-03-04T12:13:07.731515+0000","last_undegraded":"2024-03-04T12:13:07.731515+0000","last_fullsized":"2024-03-04T12:13:07.731515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.788131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.736761+0000","last_change":"2024-03-04T12:13:06.735878+0000","last_active":"2024-03-04T12:13:06.736761+0000","last_peered":"2024-03-04T12:13:06.736761+0000","last_clean":"2024-03-04T12:13:06.736761+0000","last_became_active":"2024-03-04T12:13:06.735629+0000","last_became_peered":"2024-03-04T12:13:06.735629+0000","last_unstale":"2024-03-04T12:13:06.736761+0000","last_undegraded":"2024-03-04T12:13:06.736761+0000","last_fullsized":"2024-03-04T12:13:06.736761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:57.516181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731375+0000","last_change":"2024-03-04T12:13:05.738609+0000","last_active":"2024-03-04T12:13:07.731375+0000","last_peered":"2024-03-04T12:13:07.731375+0000","last_clean":"2024-03-04T12:13:07.731375+0000","last_became_active":"2024-03-04T12:13:05.738411+0000","last_became_peered":"2024-03-04T12:13:05.738411+0000","last_unstale":"2024-03-04T12:13:07.731375+0000","last_undegraded":"2024-03-04T12:13:07.731375+0000","last_fullsized":"2024-03-04T12:13:07.731375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:26:25.947943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719736+0000","last_change":"2024-03-04T12:13:06.738677+0000","last_active":"2024-03-04T12:13:07.719736+0000","last_peered":"2024-03-04T12:13:07.719736+0000","last_clean":"2024-03-04T12:13:07.719736+0000","last_became_active":"2024-03-04T12:13:06.737413+0000","last_became_peered":"2024-03-04T12:13:06.737413+0000","last_unstale":"2024-03-04T12:13:07.719736+0000","last_undegraded":"2024-03-04T12:13:07.719736+0000","last_fullsized":"2024-03-04T12:13:07.719736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:12:58.923163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719286+0000","last_change":"2024-03-04T12:13:05.731210+0000","last_active":"2024-03-04T12:13:07.719286+0000","last_peered":"2024-03-04T12:13:07.719286+0000","last_clean":"2024-03-04T12:13:07.719286+0000","last_became_active":"2024-03-04T12:13:05.730892+0000","last_became_peered":"2024-03-04T12:13:05.730892+0000","last_unstale":"2024-03-04T12:13:07.719286+0000","last_undegraded":"2024-03-04T12:13:07.719286+0000","last_fullsized":"2024-03-04T12:13:07.719286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:15:29.358867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718243+0000","last_change":"2024-03-04T12:13:06.733212+0000","last_active":"2024-03-04T12:13:07.718243+0000","last_peered":"2024-03-04T12:13:07.718243+0000","last_clean":"2024-03-04T12:13:07.718243+0000","last_became_active":"2024-03-04T12:13:06.732967+0000","last_became_peered":"2024-03-04T12:13:06.732967+0000","last_unstale":"2024-03-04T12:13:07.718243+0000","last_undegraded":"2024-03-04T12:13:07.718243+0000","last_fullsized":"2024-03-04T12:13:07.718243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:53:28.474198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.731281+0000","last_change":"2024-03-04T12:13:05.736799+0000","last_active":"2024-03-04T12:13:07.731281+0000","last_peered":"2024-03-04T12:13:07.731281+0000","last_clean":"2024-03-04T12:13:07.731281+0000","last_became_active":"2024-03-04T12:13:05.736087+0000","last_became_peered":"2024-03-04T12:13:05.736087+0000","last_unstale":"2024-03-04T12:13:07.731281+0000","last_undegraded":"2024-03-04T12:13:07.731281+0000","last_fullsized":"2024-03-04T12:13:07.731281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:00:02.211146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.734922+0000","last_change":"2024-03-04T12:13:06.734268+0000","last_active":"2024-03-04T12:13:06.734922+0000","last_peered":"2024-03-04T12:13:06.734922+0000","last_clean":"2024-03-04T12:13:06.734922+0000","last_became_active":"2024-03-04T12:13:06.734016+0000","last_became_peered":"2024-03-04T12:13:06.734016+0000","last_unstale":"2024-03-04T12:13:06.734922+0000","last_undegraded":"2024-03-04T12:13:06.734922+0000","last_fullsized":"2024-03-04T12:13:06.734922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:14.741746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.720136+0000","last_change":"2024-03-04T12:13:05.733324+0000","last_active":"2024-03-04T12:13:07.720136+0000","last_peered":"2024-03-04T12:13:07.720136+0000","last_clean":"2024-03-04T12:13:07.720136+0000","last_became_active":"2024-03-04T12:13:05.733025+0000","last_became_peered":"2024-03-04T12:13:05.733025+0000","last_unstale":"2024-03-04T12:13:07.720136+0000","last_undegraded":"2024-03-04T12:13:07.720136+0000","last_fullsized":"2024-03-04T12:13:07.720136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:42:56.514331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719429+0000","last_change":"2024-03-04T12:13:06.744828+0000","last_active":"2024-03-04T12:13:07.719429+0000","last_peered":"2024-03-04T12:13:07.719429+0000","last_clean":"2024-03-04T12:13:07.719429+0000","last_became_active":"2024-03-04T12:13:06.742465+0000","last_became_peered":"2024-03-04T12:13:06.742465+0000","last_unstale":"2024-03-04T12:13:07.719429+0000","last_undegraded":"2024-03-04T12:13:07.719429+0000","last_fullsized":"2024-03-04T12:13:07.719429+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:41.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.733163+0000","last_change":"2024-03-04T12:13:05.734842+0000","last_active":"2024-03-04T12:13:07.733163+0000","last_peered":"2024-03-04T12:13:07.733163+0000","last_clean":"2024-03-04T12:13:07.733163+0000","last_became_active":"2024-03-04T12:13:05.734531+0000","last_became_peered":"2024-03-04T12:13:05.734531+0000","last_unstale":"2024-03-04T12:13:07.733163+0000","last_undegraded":"2024-03-04T12:13:07.733163+0000","last_fullsized":"2024-03-04T12:13:07.733163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:51:49.288101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.718228+0000","last_change":"2024-03-04T12:13:06.746153+0000","last_active":"2024-03-04T12:13:07.718228+0000","last_peered":"2024-03-04T12:13:07.718228+0000","last_clean":"2024-03-04T12:13:07.718228+0000","last_became_active":"2024-03-04T12:13:06.745897+0000","last_became_peered":"2024-03-04T12:13:06.745897+0000","last_unstale":"2024-03-04T12:13:07.718228+0000","last_undegraded":"2024-03-04T12:13:07.718228+0000","last_fullsized":"2024-03-04T12:13:07.718228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:25.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.721785+0000","last_change":"2024-03-04T12:13:05.737003+0000","last_active":"2024-03-04T12:13:06.721785+0000","last_peered":"2024-03-04T12:13:06.721785+0000","last_clean":"2024-03-04T12:13:06.721785+0000","last_became_active":"2024-03-04T12:13:05.736740+0000","last_became_peered":"2024-03-04T12:13:05.736740+0000","last_unstale":"2024-03-04T12:13:06.721785+0000","last_undegraded":"2024-03-04T12:13:06.721785+0000","last_fullsized":"2024-03-04T12:13:06.721785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:20:45.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.729158+0000","last_change":"2024-03-04T12:13:06.727366+0000","last_active":"2024-03-04T12:13:06.729158+0000","last_peered":"2024-03-04T12:13:06.729158+0000","last_clean":"2024-03-04T12:13:06.729158+0000","last_became_active":"2024-03-04T12:13:06.727120+0000","last_became_peered":"2024-03-04T12:13:06.727120+0000","last_unstale":"2024-03-04T12:13:06.729158+0000","last_undegraded":"2024-03-04T12:13:06.729158+0000","last_fullsized":"2024-03-04T12:13:06.729158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:09:25.460993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.726459+0000","last_change":"2024-03-04T12:13:05.732720+0000","last_active":"2024-03-04T12:13:06.726459+0000","last_peered":"2024-03-04T12:13:06.726459+0000","last_clean":"2024-03-04T12:13:06.726459+0000","last_became_active":"2024-03-04T12:13:05.732433+0000","last_became_peered":"2024-03-04T12:13:05.732433+0000","last_unstale":"2024-03-04T12:13:06.726459+0000","last_undegraded":"2024-03-04T12:13:06.726459+0000","last_fullsized":"2024-03-04T12:13:06.726459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:39.246078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.730822+0000","last_change":"2024-03-04T12:13:06.727385+0000","last_active":"2024-03-04T12:13:06.730822+0000","last_peered":"2024-03-04T12:13:06.730822+0000","last_clean":"2024-03-04T12:13:06.730822+0000","last_became_active":"2024-03-04T12:13:06.727076+0000","last_became_peered":"2024-03-04T12:13:06.727076+0000","last_unstale":"2024-03-04T12:13:06.730822+0000","last_undegraded":"2024-03-04T12:13:06.730822+0000","last_fullsized":"2024-03-04T12:13:06.730822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:37.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.729229+0000","last_change":"2024-03-04T12:13:05.733375+0000","last_active":"2024-03-04T12:13:06.729229+0000","last_peered":"2024-03-04T12:13:06.729229+0000","last_clean":"2024-03-04T12:13:06.729229+0000","last_became_active":"2024-03-04T12:13:05.733096+0000","last_became_peered":"2024-03-04T12:13:05.733096+0000","last_unstale":"2024-03-04T12:13:06.729229+0000","last_undegraded":"2024-03-04T12:13:06.729229+0000","last_fullsized":"2024-03-04T12:13:06.729229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:34:59.445308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.745030+0000","last_change":"2024-03-04T12:13:06.740367+0000","last_active":"2024-03-04T12:13:06.745030+0000","last_peered":"2024-03-04T12:13:06.745030+0000","last_clean":"2024-03-04T12:13:06.745030+0000","last_became_active":"2024-03-04T12:13:06.739826+0000","last_became_peered":"2024-03-04T12:13:06.739826+0000","last_unstale":"2024-03-04T12:13:06.745030+0000","last_undegraded":"2024-03-04T12:13:06.745030+0000","last_fullsized":"2024-03-04T12:13:06.745030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:40.247928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-04T12:13:06.728203+0000","last_change":"2024-03-04T12:13:05.723793+0000","last_active":"2024-03-04T12:13:06.728203+0000","last_peered":"2024-03-04T12:13:06.728203+0000","last_clean":"2024-03-04T12:13:06.728203+0000","last_became_active":"2024-03-04T12:13:05.723496+0000","last_became_peered":"2024-03-04T12:13:05.723496+0000","last_unstale":"2024-03-04T12:13:06.728203+0000","last_undegraded":"2024-03-04T12:13:06.728203+0000","last_fullsized":"2024-03-04T12:13:06.728203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:54.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719270+0000","last_change":"2024-03-04T12:13:06.740765+0000","last_active":"2024-03-04T12:13:07.719270+0000","last_peered":"2024-03-04T12:13:07.719270+0000","last_clean":"2024-03-04T12:13:07.719270+0000","last_became_active":"2024-03-04T12:13:06.740498+0000","last_became_peered":"2024-03-04T12:13:06.740498+0000","last_unstale":"2024-03-04T12:13:07.719270+0000","last_undegraded":"2024-03-04T12:13:07.719270+0000","last_fullsized":"2024-03-04T12:13:07.719270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:51.258924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.717867+0000","last_change":"2024-03-04T12:13:05.728827+0000","last_active":"2024-03-04T12:13:07.717867+0000","last_peered":"2024-03-04T12:13:07.717867+0000","last_clean":"2024-03-04T12:13:07.717867+0000","last_became_active":"2024-03-04T12:13:05.728560+0000","last_became_peered":"2024-03-04T12:13:05.728560+0000","last_unstale":"2024-03-04T12:13:07.717867+0000","last_undegraded":"2024-03-04T12:13:07.717867+0000","last_fullsized":"2024-03-04T12:13:07.717867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:45.391965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719823+0000","last_change":"2024-03-04T12:13:06.725267+0000","last_active":"2024-03-04T12:13:07.719823+0000","last_peered":"2024-03-04T12:13:07.719823+0000","last_clean":"2024-03-04T12:13:07.719823+0000","last_became_active":"2024-03-04T12:13:06.724965+0000","last_became_peered":"2024-03-04T12:13:06.724965+0000","last_unstale":"2024-03-04T12:13:07.719823+0000","last_undegraded":"2024-03-04T12:13:07.719823+0000","last_fullsized":"2024-03-04T12:13:07.719823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:05.709272+0000","last_clean_scrub_stamp":"2024-03-04T12:13:05.709272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:45:48.052770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719299+0000","last_change":"2024-03-04T12:13:05.732899+0000","last_active":"2024-03-04T12:13:07.719299+0000","last_peered":"2024-03-04T12:13:07.719299+0000","last_clean":"2024-03-04T12:13:07.719299+0000","last_became_active":"2024-03-04T12:13:05.732563+0000","last_became_peered":"2024-03-04T12:13:05.732563+0000","last_unstale":"2024-03-04T12:13:07.719299+0000","last_undegraded":"2024-03-04T12:13:07.719299+0000","last_fullsized":"2024-03-04T12:13:07.719299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:13:04.707422+0000","last_clean_scrub_stamp":"2024-03-04T12:13:04.707422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:27:13.739896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-04T12:13:07.719459+0000","last_change":"2024-03-04T12:11:43.944598+0000","last_active":"2024-03-04T12:13:07.719459+0000","last_peered":"2024-03-04T12:13:07.719459+0000","last_clean":"2024-03-04T12:13:07.719459+0000","last_became_active":"2024-03-04T12:11:43.941715+0000","last_became_peered":"2024-03-04T12:11:43.941715+0000","last_unstale":"2024-03-04T12:13:07.719459+0000","last_undegraded":"2024-03-04T12:13:07.719459+0000","last_fullsized":"2024-03-04T12:13:07.719459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T12:11:42.835630+0000","last_clean_scrub_stamp":"2024-03-04T12:11:42.835630+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:40.580229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":775386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T12:13:10.030 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-04T12:13:10.030 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-04T12:13:10.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.175+0000 7f93e0059640 1 Processor -- start 2024-03-04T12:13:10.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.175+0000 7f93e0059640 1 -- start start 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93e0059640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 0x7f93d815db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93e0059640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d806ae00 con 0x7f93d80cc900 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93e0059640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d806b0e0 con 0x7f93d80cdd70 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93e0059640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80ccde0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93dd5f4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f93d80cc900 0x7f93d815d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52222/0 (socket says 172.21.15.148:52222) 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93dd5f4640 1 -- 172.21.15.148:0/3856631406 learned_addr learned my addr 172.21.15.148:0/3856631406 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.176+0000 7f93de5f6640 1 --2- 172.21.15.148:0/3856631406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 0x7f93d815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528569508 0 0) 0x7f93d806ae00 con 0x7f93d80cc900 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c4003520 con 0x7f93d80cc900 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964141859 0 0) 0x7f93d806b0e0 con 0x7f93d80cdd70 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d806ae00 con 0x7f93d80cdd70 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483565454 0 0) 0x7f93c4003520 con 0x7f93d80cc900 2024-03-04T12:13:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93de5f6640 1 -- 172.21.15.148:0/3856631406 >> v1:172.21.15.191:6789/0 conn(0x7f93d80cdd70 legacy=0x7f93d80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93de5f6640 1 -- 172.21.15.148:0/3856631406 >> v1:172.21.15.148:6789/0 conn(0x7f93d80cc900 legacy=0x7f93d815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.177+0000 7f93de5f6640 1 -- 172.21.15.148:0/3856631406 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d806b3c0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93de5f6640 1 --2- 172.21.15.148:0/3856631406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 0x7f93d815db20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009dc0 tx=0x7f93d002bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd9f033daeccc9a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0039070 con 0x7f93d80ccfe0 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d002c880 con 0x7f93d80ccfe0 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93dcdf3640 1 -- 172.21.15.148:0/3856631406 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8001670 con 0x7f93d80ccfe0 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93e0059640 1 -- 172.21.15.148:0/3856631406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 msgr2=0x7f93d815db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93e0059640 1 --2- 172.21.15.148:0/3856631406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 0x7f93d815db20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009dc0 tx=0x7f93d002bd50 comp rx=0 tx=0).stop 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93e0059640 1 -- 172.21.15.148:0/3856631406 shutdown_connections 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93e0059640 1 --2- 172.21.15.148:0/3856631406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80ccfe0 0x7f93d815db20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.178+0000 7f93e0059640 1 -- 172.21.15.148:0/3856631406 >> 172.21.15.148:0/3856631406 conn(0x7f93d80b22e0 msgr2=0x7f93d80b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.179+0000 7f93e0059640 1 -- 172.21.15.148:0/3856631406 shutdown_connections 2024-03-04T12:13:10.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.179+0000 7f93e0059640 1 -- 172.21.15.148:0/3856631406 wait complete. 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 Processor -- start 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 -- start start 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 0x7f93d8154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 0x7f93d80b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d815e1e0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d815e060 con 0x7f93d80cc900 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93e0059640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b2e90 con 0x7f93d80cdd70 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93dd5f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93dd5f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53668/0 (socket says 172.21.15.148:53668) 2024-03-04T12:13:10.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.180+0000 7f93dd5f4640 1 -- 172.21.15.148:0/1255165689 learned_addr learned my addr 172.21.15.148:0/1255165689 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dddf5640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 0x7f93d8154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93de5f6640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 0x7f93d80b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 -- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 msgr2=0x7f93d80b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 0x7f93d80b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 -- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 msgr2=0x7f93d8154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 0x7f93d8154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 -- 172.21.15.148:0/1255165689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80b44c0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dddf5640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 0x7f93d8154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93de5f6640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 0x7f93d80b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93dd5f4640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f93c800c750 tx=0x7f93c800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8005ad0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8005c70 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d80b47a0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8007290 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.181+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d80b4d60 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.182+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d80cc480 con 0x7f93d80ccfe0 2024-03-04T12:13:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.182+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f93c8007430 con 0x7f93d80ccfe0 2024-03-04T12:13:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.183+0000 7f93ce7fc640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.183+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93c808a1a0 con 0x7f93d80ccfe0 2024-03-04T12:13:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.183+0000 7f93dddf5640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.183+0000 7f93dddf5640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 0x7f93bc06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93d40011e0 tx=0x7f93d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.187+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93d80cc480 con 0x7f93d80ccfe0 2024-03-04T12:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.423+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f93d80b4f60 con 0x7f93d80ccfe0 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.425+0000 7f93ce7fc640 1 -- 172.21.15.148:0/1255165689 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f93d80b4f60 con 0x7f93d80ccfe0 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBRuuVlHlfkBBAAcGWanX3ahyQy3t9Q4LBHWQ== 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-03-04T12:13:10.427 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-04T12:13:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 msgr2=0x7f93bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 0x7f93bc06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93d40011e0 tx=0x7f93d4009040 comp rx=0 tx=0).stop 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 msgr2=0x7f93d80ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f93c800c750 tx=0x7f93c800cc10 comp rx=0 tx=0).stop 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 shutdown_connections 2024-03-04T12:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93d80cdd70 0x7f93d80b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93d80ccfe0 0x7f93d80ca260 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 --2- 172.21.15.148:0/1255165689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93d80cc900 0x7f93d8154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 >> 172.21.15.148:0/1255165689 conn(0x7f93d80b22e0 msgr2=0x7f93d80b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 shutdown_connections 2024-03-04T12:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.427+0000 7f93e0059640 1 -- 172.21.15.148:0/1255165689 wait complete. 2024-03-04T12:13:10.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T12:13:10.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.538+0000 7f0c11990640 1 Processor -- start 2024-03-04T12:13:10.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.538+0000 7f0c11990640 1 -- start start 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c11990640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 0x7f0c0c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c11990640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0c0c0cc8e0 0x7f0c0c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c11990640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0cb300 con 0x7f0c0c0d1290 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c11990640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d0400 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c11990640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c06ae50 con 0x7f0c0c0cfae0 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c0b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0c0c0d1290 0x7f0c0c0cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52236/0 (socket says 172.21.15.148:52236) 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.539+0000 7f0c0b7fe640 1 -- 172.21.15.148:0/3233074318 learned_addr learned my addr 172.21.15.148:0/3233074318 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0a7fc640 1 --2- 172.21.15.148:0/3233074318 >> v2:172.21.15.191:3300/0 conn(0x7f0c0c0cc8e0 0x7f0c0c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0affd640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 0x7f0c0c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568185139 0 0) 0x7f0c0c0cb300 con 0x7f0c0c0d1290 2024-03-04T12:13:10.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf4003520 con 0x7f0c0c0d1290 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341842470 0 0) 0x7f0bf4003520 con 0x7f0c0c0d1290 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0a7fc640 1 -- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 msgr2=0x7f0c0c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0a7fc640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 0x7f0c0c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0a7fc640 1 -- 172.21.15.148:0/3233074318 >> v1:172.21.15.148:6789/0 conn(0x7f0c0c0d1290 legacy=0x7f0c0c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0a7fc640 1 -- 172.21.15.148:0/3233074318 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c06b130 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.540+0000 7f0c0affd640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 0x7f0c0c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:13:10.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c0a7fc640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cc8e0 0x7f0c0c0cfec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc80afb517201da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00037070 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00030510 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c09ffb640 1 -- 172.21.15.148:0/3233074318 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04001670 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 -- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cc8e0 msgr2=0x7f0c0c0cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cc8e0 0x7f0c0c0cfec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b790 comp rx=0 tx=0).stop 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 -- 172.21.15.148:0/3233074318 shutdown_connections 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cc8e0 0x7f0c0c0cfec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 --2- 172.21.15.148:0/3233074318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cfae0 0x7f0c0c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 -- 172.21.15.148:0/3233074318 >> 172.21.15.148:0/3233074318 conn(0x7f0c0c0d0770 msgr2=0x7f0c0c0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.541+0000 7f0c11990640 1 -- 172.21.15.148:0/3233074318 shutdown_connections 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.542+0000 7f0c11990640 1 -- 172.21.15.148:0/3233074318 wait complete. 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 Processor -- start 2024-03-04T12:13:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 -- start start 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cc8e0 0x7f0c0c1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cfae0 0x7f0c0c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0ce440 con 0x7f0c0c0d1290 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c06b6f0 con 0x7f0c0c0cfae0 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.543+0000 7f0c11990640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c06b9d0 con 0x7f0c0c0cc8e0 2024-03-04T12:13:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53670/0 (socket says 172.21.15.148:53670) 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 -- 172.21.15.148:0/2479061149 learned_addr learned my addr 172.21.15.148:0/2479061149 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 -- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cc8e0 msgr2=0x7f0c0c1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cc8e0 0x7f0c0c1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 -- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cfae0 msgr2=0x7f0c0c0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cfae0 0x7f0c0c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 -- 172.21.15.148:0/2479061149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0b3370 con 0x7f0c0c0d1290 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c0b7fe640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c750 tx=0x7f0c0400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005ad0 con 0x7f0c0c0d1290 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04005c90 con 0x7f0c0c0d1290 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0b3650 con 0x7f0c0c0d1290 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c040072b0 con 0x7f0c0c0d1290 2024-03-04T12:13:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.544+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0b3c10 con 0x7f0c0c0d1290 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.545+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c074f60 con 0x7f0c0c0d1290 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.546+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c04007450 con 0x7f0c0c0d1290 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.546+0000 7f0bf3fff640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.546+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c0408a210 con 0x7f0c0c0d1290 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.546+0000 7f0c0affd640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:10.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.547+0000 7f0c0affd640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c15f2d0 tx=0x7f0bfc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.550+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c04058f00 con 0x7f0c0c0d1290 2024-03-04T12:13:10.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.758+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c0c0ceb30 con 0x7f0c0c0d1290 2024-03-04T12:13:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.758+0000 7f0bf3fff640 1 -- 172.21.15.148:0/2479061149 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f0c04058800 con 0x7f0c0c0d1290 2024-03-04T12:13:10.760 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:10.760 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3112149074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3112149074},{"type":"v1","addr":"172.21.15.191:6835","nonce":3112149074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3334626157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3334626157},{"type":"v1","addr":"172.21.15.191:6834","nonce":3334626157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2949030514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2949030514},{"type":"v1","addr":"172.21.15.148:6836","nonce":2949030514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T12:13:06.060663+0000","modified":"2024-03-04T12:13:08.064737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5346":{"gid":5346,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/130422954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":130422954},{"type":"v1","addr":"172.21.15.148:6837","nonce":130422954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-04T12:13:10.761 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 15 2024-03-04T12:13:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 msgr2=0x7f0bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c15f2d0 tx=0x7f0bfc00b040 comp rx=0 tx=0).stop 2024-03-04T12:13:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 msgr2=0x7f0c0c0b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c750 tx=0x7f0c0400cc10 comp rx=0 tx=0).stop 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 shutdown_connections 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c0c0d1290 0x7f0c0c0b5580 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c0c0cfae0 0x7f0c0c0b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 --2- 172.21.15.148:0/2479061149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c0c0cc8e0 0x7f0c0c1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 >> 172.21.15.148:0/2479061149 conn(0x7f0c0c0d0770 msgr2=0x7f0c0c0c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 shutdown_connections 2024-03-04T12:13:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:10.760+0000 7f0c11990640 1 -- 172.21.15.148:0/2479061149 wait complete. 2024-03-04T12:13:11.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T12:13:06.060663+0000', 'modified': '2024-03-04T12:13:08.064737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5346}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5346': {'gid': 5346, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/130422954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 130422954}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 130422954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-04T12:13:11.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T12:13:11.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-04T12:13:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.277+0000 7f8ae1da9640 1 Processor -- start 2024-03-04T12:13:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.277+0000 7f8ae1da9640 1 -- start start 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.278+0000 7f8ae1da9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.278+0000 7f8ae1da9640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8adc0b5470 0x7f8adc0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.278+0000 7f8ae1da9640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc15e1e0 con 0x7f8adc15de00 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.278+0000 7f8ae1da9640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0b98c0 con 0x7f8adc0b5470 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.278+0000 7f8ae1da9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0b32c0 con 0x7f8adc0b2e60 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adaffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adbfff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8adc0b5470 0x7f8adc0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adb7fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8adc15de00 0x7f8adc0b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:39532/0 (socket says 172.21.15.148:39532) 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adb7fe640 1 -- 172.21.15.148:0/893853358 learned_addr learned my addr 172.21.15.148:0/893853358 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adbfff640 1 -- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 msgr2=0x7f8adc0b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adbfff640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adbfff640 1 -- 172.21.15.148:0/893853358 >> v1:172.21.15.191:6789/0 conn(0x7f8adc15de00 legacy=0x7f8adc0b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adbfff640 1 -- 172.21.15.148:0/893853358 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0d1c10 con 0x7f8adc0b5470 2024-03-04T12:13:11.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8ada7fc640 1 -- 172.21.15.148:0/893853358 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881092732 0 0) 0x7f8adc15e1e0 con 0x7f8adc15de00 2024-03-04T12:13:11.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.279+0000 7f8adaffd640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T12:13:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8adbfff640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0b5900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009d90 tx=0x7f8ad402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a36254c68d5b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ada7fc640 1 -- 172.21.15.148:0/893853358 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4039070 con 0x7f8adc0b5470 2024-03-04T12:13:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ada7fc640 1 -- 172.21.15.148:0/893853358 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad402cc50 con 0x7f8adc0b5470 2024-03-04T12:13:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ada7fc640 1 -- 172.21.15.148:0/893853358 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4034440 con 0x7f8adc0b5470 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 -- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 msgr2=0x7f8adc0b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0b5900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009d90 tx=0x7f8ad402bea0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 -- 172.21.15.148:0/893853358 shutdown_connections 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0b5900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/893853358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.280+0000 7f8ae1da9640 1 -- 172.21.15.148:0/893853358 >> 172.21.15.148:0/893853358 conn(0x7f8adc0d0470 msgr2=0x7f8adc0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.281+0000 7f8ae1da9640 1 -- 172.21.15.148:0/893853358 shutdown_connections 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.281+0000 7f8ae1da9640 1 -- 172.21.15.148:0/893853358 wait complete. 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 Processor -- start 2024-03-04T12:13:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 -- start start 2024-03-04T12:13:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0d21d0 con 0x7f8adc15de00 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.282+0000 7f8ae1da9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0d2820 con 0x7f8adc0b5470 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ae1da9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0d24b0 con 0x7f8adc0b2e60 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53686/0 (socket says 172.21.15.148:53686) 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 -- 172.21.15.148:0/1645518978 learned_addr learned my addr 172.21.15.148:0/1645518978 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adaffd640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 -- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 msgr2=0x7f8adc0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 -- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 msgr2=0x7f8adc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 -- 172.21.15.148:0/1645518978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0cb4d0 con 0x7f8adc15de00 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4037040 tx=0x7f8ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ac3fff640 1 -- 172.21.15.148:0/1645518978 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4044020 con 0x7f8adc15de00 2024-03-04T12:13:11.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc0aa3d0 con 0x7f8adc15de00 2024-03-04T12:13:11.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ac3fff640 1 -- 172.21.15.148:0/1645518978 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad403a040 con 0x7f8adc15de00 2024-03-04T12:13:11.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ac3fff640 1 -- 172.21.15.148:0/1645518978 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4039050 con 0x7f8adc15de00 2024-03-04T12:13:11.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.283+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0aa990 con 0x7f8adc15de00 2024-03-04T12:13:11.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.285+0000 7f8ac3fff640 1 -- 172.21.15.148:0/1645518978 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8ad4003990 con 0x7f8adc15de00 2024-03-04T12:13:11.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.285+0000 7f8ac3fff640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.285+0000 7f8ac3fff640 1 -- 172.21.15.148:0/1645518978 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad40ac640 con 0x7f8adc15de00 2024-03-04T12:13:11.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.285+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.286+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 0x7f8ac806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000f60 tx=0x7f8acc007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.286+0000 7f8ae4989740 1 Processor -- start 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.286+0000 7f8ae4989740 1 -- start start 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x5616669e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616669237c0 con 0x5616669b9380 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616668ccb50 con 0x5616669e6280 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x56166691f860 con 0x5616669e5ea0 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53700/0 (socket says 172.21.15.148:53700) 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 -- 172.21.15.148:0/2655182934 learned_addr learned my addr 172.21.15.148:0/2655182934 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adaffd640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x5616669e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 -- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 msgr2=0x5616669b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 -- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 msgr2=0x5616669e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x5616669e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.287+0000 7f8adbfff640 1 -- 172.21.15.148:0/2655182934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616668e09a0 con 0x5616669b9380 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8adaffd640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x5616669e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40aa9e0 tx=0x7f8ad4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e7be92a38c51c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8ac17fa640 1 -- 172.21.15.148:0/2655182934 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4081070 con 0x5616669b9380 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8ac17fa640 1 -- 172.21.15.148:0/2655182934 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad407ed90 con 0x5616669b9380 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8ac17fa640 1 -- 172.21.15.148:0/2655182934 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4083350 con 0x5616669b9380 2024-03-04T12:13:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8ae4989740 1 -- 172.21.15.148:0/2655182934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 msgr2=0x5616669e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.288+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40aa9e0 tx=0x7f8ad4009a50 comp rx=0 tx=0).stop 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 -- 172.21.15.148:0/2655182934 shutdown_connections 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669b9380 0x5616669e9170 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x5616669e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2655182934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 -- 172.21.15.148:0/2655182934 >> 172.21.15.148:0/2655182934 conn(0x5616669bb0b0 msgr2=0x561666a03c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 -- 172.21.15.148:0/2655182934 shutdown_connections 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 -- 172.21.15.148:0/2655182934 wait complete. 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 -- start start 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.289+0000 7f8ae4989740 1 Processor -- start 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561666a04220 0x5616669ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616669bd750 con 0x5616669e6280 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616668dff90 con 0x561666a04220 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616669bff90 con 0x5616669e5ea0 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8ae4989740 10 client.0 _ll_register_callbacks cb 0x561666901d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53702/0 (socket says 172.21.15.148:53702) 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 -- 172.21.15.148:0/3765288603 learned_addr learned my addr 172.21.15.148:0/3765288603 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adbfff640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561666a04220 0x5616669ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 -- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 msgr2=0x5616669eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 -- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561666a04220 msgr2=0x5616669ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561666a04220 0x5616669ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adaffd640 1 -- 172.21.15.148:0/3765288603 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616666382d0 con 0x5616669e6280 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.290+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T12:13:11.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8adaffd640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8adc161460 tx=0x7f8ad0014ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27529d0ca43f76bf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0004300 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad0003610 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad407ed90 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad0027050 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 10 client.5526 fetch_fsmap learned FSMap version 15 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5616667026b0 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad407ed90 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ad002a020 con 0x5616669e6280 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 10 client.5526 fetch_fsmap finished waiting for FSMap version 15 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 10 client.5526 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 20 client.5526 populate_metadata read hostname 'smithi148' 2024-03-04T12:13:11.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a17ad0 0x561666a17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 4 client.5526 mds_command: new command op to 5346 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 --> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561666930d00 con 0x561666a17ad0 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.291+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a17ad0 0x561666a17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a17ad0 0x561666a17eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ac72722a6d5e1 server_cookie=8ea2af61fac8362e in_seq=0 out_seq=0 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8aba7fc640 10 client.5526 ms_handle_connect on v2:172.21.15.148:6835/130422954 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8aba7fc640 1 -- 172.21.15.148:0/3765288603 <== mds.0 v2:172.21.15.148:6835/130422954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561666930d00 con 0x561666a17ad0 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8aba7fc640 10 client.5526 handle_command_reply: tid=0 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 client.5526 shutdown 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 msgr2=0x5616669ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8adc161460 tx=0x7f8ad0014ae0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 shutdown_connections 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561666a04220 0x5616669ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616669e6280 0x5616669ebae0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a17ad0 0x561666a17eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 --2- 172.21.15.148:0/3765288603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x5616669eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 >> 172.21.15.148:0/3765288603 conn(0x5616669b9380 msgr2=0x5616669bce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8aba7fc640 0 client.5526 ms_handle_reset on v2:172.21.15.148:6835/130422954 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 shutdown_connections 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.292+0000 7f8ae4989740 1 -- 172.21.15.148:0/3765288603 wait complete. 2024-03-04T12:13:11.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.293+0000 7f8ae4989740 20 client.5526 trim_cache size 0 max 16384 2024-03-04T12:13:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.308+0000 7f8ae4989740 1 Processor -- start 2024-03-04T12:13:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.308+0000 7f8ae4989740 1 -- start start 2024-03-04T12:13:11.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 0x5616669e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e6280 0x5616669d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc2003c0 con 0x561666a04220 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1ff520 con 0x5616669e5ea0 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1fe920 con 0x5616669e6280 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53710/0 (socket says 172.21.15.148:53710) 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 -- 172.21.15.148:0/1536402164 learned_addr learned my addr 172.21.15.148:0/1536402164 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 0x5616669e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adaffd640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e6280 0x5616669d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 -- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e6280 msgr2=0x5616669d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e6280 0x5616669d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 -- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 msgr2=0x5616669e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 0x5616669e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 -- 172.21.15.148:0/1536402164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561666a17200 con 0x561666a04220 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 0x5616669e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.309+0000 7f8adbfff640 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4077150 tx=0x7f8ad407d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204dea07b548211a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ac0ff9640 1 -- 172.21.15.148:0/1536402164 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4076f90 con 0x561666a04220 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ac0ff9640 1 -- 172.21.15.148:0/1536402164 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4081070 con 0x561666a04220 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ac0ff9640 1 -- 172.21.15.148:0/1536402164 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4083350 con 0x561666a04220 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 -- 172.21.15.148:0/1536402164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 msgr2=0x5616669d6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4077150 tx=0x7f8ad407d000 comp rx=0 tx=0).stop 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 -- 172.21.15.148:0/1536402164 shutdown_connections 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x5616669d6c70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e6280 0x5616669d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 --2- 172.21.15.148:0/1536402164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e5ea0 0x5616669e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 -- 172.21.15.148:0/1536402164 >> 172.21.15.148:0/1536402164 conn(0x5616669b9380 msgr2=0x561666a060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 -- 172.21.15.148:0/1536402164 shutdown_connections 2024-03-04T12:13:11.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.310+0000 7f8ae4989740 1 -- 172.21.15.148:0/1536402164 wait complete. 2024-03-04T12:13:11.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.311+0000 7f8ae4989740 1 -- start start 2024-03-04T12:13:11.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.311+0000 7f8ae4989740 1 Processor -- start 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x561666a16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x561666a168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616669edf00 con 0x561666a04220 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1fef20 con 0x5616669e6280 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616669ee150 con 0x5616669e5ea0 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 10 client.0 _ll_register_callbacks cb 0x7f8ad407aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53712/0 (socket says 172.21.15.148:53712) 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 -- 172.21.15.148:0/2995226359 learned_addr learned my addr 172.21.15.148:0/2995226359 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x561666a16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adaffd640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x561666a168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 -- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 msgr2=0x561666a16370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x561666a16370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 -- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 msgr2=0x561666a168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x561666a168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 -- 172.21.15.148:0/2995226359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561666a060a0 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8adbfff640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ad402c4b0 tx=0x7f8ad4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151128165e797144 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8aba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad409f070 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad4082ec0 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad4081070 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad4076f90 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 10 client.5532 fetch_fsmap learned FSMap version 15 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.312+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x561666a060a0 con 0x561666a04220 2024-03-04T12:13:11.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4081070 con 0x561666a04220 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ad4099b90 con 0x561666a04220 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 10 client.5532 fetch_fsmap finished waiting for FSMap version 15 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 10 client.5532 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 20 client.5532 populate_metadata read hostname 'smithi148' 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a68ec0 0x561666a6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 4 client.5532 mds_command: new command op to 5346 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 --> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56166692b050 con 0x561666a68ec0 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a68ec0 0x561666a6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8adb7fe640 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a68ec0 0x561666a6f670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6acfe79347561c server_cookie=d3140fc2c7e0c05f in_seq=0 out_seq=0 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 10 client.5532 ms_handle_connect on v2:172.21.15.148:6835/130422954 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 1 -- 172.21.15.148:0/2995226359 <== mds.0 v2:172.21.15.148:6835/130422954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56166692b050 con 0x561666a68ec0 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 10 client.5532 handle_command_reply: tid=0 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 client.5532 shutdown 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 msgr2=0x561666a6f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ad402c4b0 tx=0x7f8ad4009750 comp rx=0 tx=0).stop 2024-03-04T12:13:11.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 shutdown_connections 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561666a04220 0x561666a6f130 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5616669e6280 0x561666a168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.148:6835/130422954,v1:172.21.15.148:6837/130422954] conn(0x561666a68ec0 0x561666a6f670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 --2- 172.21.15.148:0/2995226359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5616669e5ea0 0x561666a16370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8aba7fc640 0 client.5532 ms_handle_reset on v2:172.21.15.148:6835/130422954 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.313+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 >> 172.21.15.148:0/2995226359 conn(0x5616669b9380 msgr2=0x5616669bebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 shutdown_connections 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae4989740 1 -- 172.21.15.148:0/2995226359 wait complete. 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae4989740 20 client.5532 trim_cache size 0 max 16384 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "d4b3ed31-a94c-42d0-968e-1d6a4e199a28", 2024-03-04T12:13:11.322 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5346, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5898203210000004, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 15, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 38, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 38, 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 11.664925529 2024-03-04T12:13:11.323 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 msgr2=0x7f8ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 0x7f8ac806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000f60 tx=0x7f8acc007d50 comp rx=0 tx=0).stop 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 msgr2=0x7f8adc0cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4037040 tx=0x7f8ad4004a40 comp rx=0 tx=0).stop 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 shutdown_connections 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc15de00 0x7f8adc0cc9f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc0b5470 0x7f8adc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 --2- 172.21.15.148:0/1645518978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8adc0b2e60 0x7f8adc0cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 >> 172.21.15.148:0/1645518978 conn(0x7f8adc0d0470 msgr2=0x7f8adc0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 shutdown_connections 2024-03-04T12:13:11.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.314+0000 7f8ae1da9640 1 -- 172.21.15.148:0/1645518978 wait complete. 2024-03-04T12:13:11.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T12:13:11.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.425+0000 7fe42b9ab640 1 Processor -- start 2024-03-04T12:13:11.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.426+0000 7fe42b9ab640 1 -- start start 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe42b9ab640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe42415de00 0x7fe4240b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe42b9ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe42b9ab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42415e1e0 con 0x7fe4240b9380 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe42b9ab640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4240d2820 con 0x7fe42415de00 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe42b9ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d1c10 con 0x7fe4240b5470 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe42415de00 0x7fe4240b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe42415de00 0x7fe4240b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53716/0 (socket says 172.21.15.148:53716) 2024-03-04T12:13:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 -- 172.21.15.148:0/2956012822 learned_addr learned my addr 172.21.15.148:0/2956012822 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429f48640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 -- 172.21.15.148:0/2956012822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 msgr2=0x7fe4240b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:11.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 -- 172.21.15.148:0/2956012822 >> v1:172.21.15.148:6789/0 conn(0x7fe4240b9380 legacy=0x7fe4240b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.427+0000 7fe429747640 1 -- 172.21.15.148:0/2956012822 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240d1ef0 con 0x7fe42415de00 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.428+0000 7fe429747640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b2e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe420009870 tx=0x7fe42002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b30237bde3b45d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.428+0000 7fe41bfff640 1 -- 172.21.15.148:0/2956012822 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420034070 con 0x7fe42415de00 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.428+0000 7fe41bfff640 1 -- 172.21.15.148:0/2956012822 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420030840 con 0x7fe42415de00 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.428+0000 7fe41bfff640 1 -- 172.21.15.148:0/2956012822 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420030c10 con 0x7fe42415de00 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 -- 172.21.15.148:0/2956012822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 msgr2=0x7fe4240b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b2e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe420009870 tx=0x7fe42002b980 comp rx=0 tx=0).stop 2024-03-04T12:13:11.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 -- 172.21.15.148:0/2956012822 shutdown_connections 2024-03-04T12:13:11.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/2956012822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b2e60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 -- 172.21.15.148:0/2956012822 >> 172.21.15.148:0/2956012822 conn(0x7fe4240d0470 msgr2=0x7fe4240d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 -- 172.21.15.148:0/2956012822 shutdown_connections 2024-03-04T12:13:11.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.429+0000 7fe42b9ab640 1 -- 172.21.15.148:0/2956012822 wait complete. 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.430+0000 7fe42b9ab640 1 Processor -- start 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.430+0000 7fe42b9ab640 1 -- start start 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4240b9380 0x7fe424160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240b97e0 con 0x7fe42415de00 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d24b0 con 0x7fe4240b9380 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240b72a0 con 0x7fe4240b5470 2024-03-04T12:13:11.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53732/0 (socket says 172.21.15.148:53732) 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 -- 172.21.15.148:0/1626800721 learned_addr learned my addr 172.21.15.148:0/1626800721 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429747640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe428f46640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4240b9380 0x7fe424160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 -- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 msgr2=0x7fe4240c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 -- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4240b9380 msgr2=0x7fe424160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4240b9380 0x7fe424160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 -- 172.21.15.148:0/1626800721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240c91b0 con 0x7fe42415de00 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe429f48640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00efc0 tx=0x7fe41c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c018020 con 0x7fe42415de00 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c014480 con 0x7fe42415de00 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4240c9490 con 0x7fe42415de00 2024-03-04T12:13:11.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.431+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c014870 con 0x7fe42415de00 2024-03-04T12:13:11.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.432+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4240c9a50 con 0x7fe42415de00 2024-03-04T12:13:11.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.433+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe424074f60 con 0x7fe42415de00 2024-03-04T12:13:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.433+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe41c005d10 con 0x7fe42415de00 2024-03-04T12:13:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.433+0000 7fe419ffb640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.434+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe41c08ace0 con 0x7fe42415de00 2024-03-04T12:13:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.434+0000 7fe429747640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.434+0000 7fe429747640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe420036000 tx=0x7fe42003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:11.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.438+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe41c022070 con 0x7fe42415de00 2024-03-04T12:13:11.511 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T12:13:11.509+0000 7fa25998b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T12:13:11.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.645+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4240b78c0 con 0x7fe42415de00 2024-03-04T12:13:11.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.645+0000 7fe419ffb640 1 -- 172.21.15.148:0/1626800721 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe41c010040 con 0x7fe42415de00 2024-03-04T12:13:11.647 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 msgr2=0x7fe40806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe420036000 tx=0x7fe42003b040 comp rx=0 tx=0).stop 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 msgr2=0x7fe4240b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00efc0 tx=0x7fe41c00c490 comp rx=0 tx=0).stop 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 shutdown_connections 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42415de00 0x7fe4240b5900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4240b9380 0x7fe424160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 --2- 172.21.15.148:0/1626800721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4240b5470 0x7fe4240c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:11.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 >> 172.21.15.148:0/1626800721 conn(0x7fe4240d0470 msgr2=0x7fe4240b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:11.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 shutdown_connections 2024-03-04T12:13:11.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:11.647+0000 7fe42b9ab640 1 -- 172.21.15.148:0/1626800721 wait complete. 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T12:13:11.660 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T12:13:11.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.661 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:13:11.676 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42650sec preferred_lft 42650sec 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9e:79:ab:19:e0:f5 brd ff:ff:ff:ff:ff:ff 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::5c1c:10ff:feb4:9a30/64 scope link 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.677 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether da:49:f7:74:bb:9a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::d849:f7ff:fe74:bb9a/64 scope link 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9e:79:ab:19:e0:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::9c79:abff:fe19:e0f5/64 scope link 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-04T12:13:11.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.678 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T12:13:11.741 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-04T12:13:11.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.743 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-04T12:13:11.799 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-04T12:13:11.799 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T12:13:11.799 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T12:13:11.799 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T12:13:11.800 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T12:13:11.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.800 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-04T12:13:11.800 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-04T12:13:11.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T12:13:11.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T12:13:11.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T12:13:11.801 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-04T12:13:11.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.895 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-04T12:13:11.909 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-04T12:13:11.910 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-04T12:13:11.910 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T12:13:11.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:11.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-04T12:13:11.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:13:11.978 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T12:13:12.033 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T12:13:12.033 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T12:13:12.033 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T12:13:12.088 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-04T12:13:12.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:12.088 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:13:12.173 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:13:12.173 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:13:12.180 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T12:13:12.180 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T12:13:12.180 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@d4b3ed31-a94c-42d0-968e-1d6a4e199a28.cephfs=/ 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T12:13:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T12:13:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T12:13:12.183 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T12:13:12.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:12.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T12:13:12.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:12.210 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T12:13:12.271 INFO:teuthology.orchestra.run.smithi148.stdout:client4934 2024-03-04T12:13:12.271 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:13:12.271 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T12:13:12.327 DEBUG:teuthology.orchestra.run.smithi148:> mktemp -d --suffix=xfstests-dev 2024-03-04T12:13:12.381 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.ATjWjXU3R3xfstests-dev 2024-03-04T12:13:12.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:12.381 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.ATjWjXU3R3xfstests-dev) 2024-03-04T12:13:12.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:12.450 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.ATjWjXU3R3xfstests-dev) 2024-03-04T12:13:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:Cloning into '/tmp/tmp.ATjWjXU3R3xfstests-dev'... 2024-03-04T12:13:13.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:13.206 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-03-04T12:13:13.265 DEBUG:teuthology.orchestra.run.smithi148:> mktemp -d --suffix=test 2024-03-04T12:13:13.320 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.1jHsm6tBGxtest 2024-03-04T12:13:13.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:13:13.321 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-03-04T12:13:13.376 DEBUG:teuthology.orchestra.run.smithi148:> mktemp -d --suffix=scratch 2024-03-04T12:13:13.431 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:13:13.431 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-03-04T12:13:13.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-03-04T12:13:14.038 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:03:47 ago on Mon 04 Mar 2024 12:09:27 PM UTC. 2024-03-04T12:13:14.153 INFO:teuthology.orchestra.run.smithi148.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-03-04T12:13:14.154 INFO:teuthology.orchestra.run.smithi148.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T12:13:14.155 INFO:teuthology.orchestra.run.smithi148.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-03-04T12:13:14.156 INFO:teuthology.orchestra.run.smithi148.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-03-04T12:13:14.157 INFO:teuthology.orchestra.run.smithi148.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-03-04T12:13:14.159 INFO:teuthology.orchestra.run.smithi148.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-03-04T12:13:14.161 INFO:teuthology.orchestra.run.smithi148.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-03-04T12:13:14.162 INFO:teuthology.orchestra.run.smithi148.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-03-04T12:13:14.164 INFO:teuthology.orchestra.run.smithi148.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-03-04T12:13:14.168 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-03-04T12:13:14.168 INFO:teuthology.orchestra.run.smithi148.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-03-04T12:13:14.170 INFO:teuthology.orchestra.run.smithi148.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-03-04T12:13:14.171 INFO:teuthology.orchestra.run.smithi148.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-03-04T12:13:14.172 INFO:teuthology.orchestra.run.smithi148.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-03-04T12:13:14.173 INFO:teuthology.orchestra.run.smithi148.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-03-04T12:13:14.174 INFO:teuthology.orchestra.run.smithi148.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-03-04T12:13:14.175 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-03-04T12:13:14.176 INFO:teuthology.orchestra.run.smithi148.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-03-04T12:13:14.177 INFO:teuthology.orchestra.run.smithi148.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-03-04T12:13:14.180 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-04T12:13:14.245 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-03-04T12:13:14.249 INFO:teuthology.orchestra.run.smithi148.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-devel x86_64 5.14.0-427.el9 appstream 22 M 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: cpp x86_64 11.4.1-3.el9 CentOS-AppStream 11 M 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper x86_64 9:1.02.197-2.el9 CentOS-BaseOS 139 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 CentOS-BaseOS 33 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 CentOS-BaseOS 31 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 CentOS-BaseOS 178 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: e2fsprogs x86_64 1.46.5-5.el9 CentOS-BaseOS 1.0 M 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 CentOS-BaseOS 221 k 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: gcc x86_64 11.4.1-3.el9 CentOS-AppStream 32 M 2024-03-04T12:13:14.250 INFO:teuthology.orchestra.run.smithi148.stdout: gcc-c++ x86_64 11.4.1-3.el9 CentOS-AppStream 13 M 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 CentOS-AppStream 46 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: git x86_64 2.43.0-1.el9 CentOS-AppStream 50 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: git-core x86_64 2.43.0-1.el9 CentOS-AppStream 4.4 M 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: git-core-doc noarch 2.43.0-1.el9 CentOS-AppStream 2.6 M 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err x86_64 1.46.5-5.el9 CentOS-BaseOS 25 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libgcc x86_64 11.4.1-3.el9 CentOS-BaseOS 91 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libgomp x86_64 11.4.1-3.el9 CentOS-BaseOS 267 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libss x86_64 1.46.5-5.el9 CentOS-BaseOS 31 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libstdc++ x86_64 11.4.1-3.el9 CentOS-BaseOS 739 k 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: libstdc++-devel x86_64 11.4.1-3.el9 CentOS-AppStream 2.2 M 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: lvm2 x86_64 9:2.03.23-2.el9 CentOS-BaseOS 1.5 M 2024-03-04T12:13:14.251 INFO:teuthology.orchestra.run.smithi148.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 CentOS-BaseOS 1.0 M 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: make x86_64 1:4.3-8.el9 CentOS-BaseOS 529 k 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: perl-Git noarch 2.43.0-1.el9 CentOS-AppStream 37 k 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-03-04T12:13:14.252 INFO:teuthology.orchestra.run.smithi148.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: libnbd x86_64 1.18.1-3.el9 CentOS-AppStream 165 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-03-04T12:13:14.253 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-04T12:13:14.254 INFO:teuthology.orchestra.run.smithi148.stdout:Install 17 Packages 2024-03-04T12:13:14.255 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 28 Packages 2024-03-04T12:13:14.255 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:14.256 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 108 M 2024-03-04T12:13:14.256 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-04T12:13:15.204 INFO:teuthology.orchestra.run.smithi148.stdout:(1/45): fio-engine-http-3.35-1.el9.x86_64.rpm 61 kB/s | 17 kB 00:00 2024-03-04T12:13:15.229 INFO:teuthology.orchestra.run.smithi148.stdout:(2/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 48 kB/s | 14 kB 00:00 2024-03-04T12:13:15.347 INFO:teuthology.orchestra.run.smithi148.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 95 kB/s | 13 kB 00:00 2024-03-04T12:13:15.372 INFO:teuthology.orchestra.run.smithi148.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 103 kB/s | 15 kB 00:00 2024-03-04T12:13:15.514 INFO:teuthology.orchestra.run.smithi148.stdout:(5/45): fio-3.35-1.el9.x86_64.rpm 11 MB/s | 6.2 MB 00:00 2024-03-04T12:13:15.530 INFO:teuthology.orchestra.run.smithi148.stdout:(6/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 82 kB/s | 15 kB 00:00 2024-03-04T12:13:15.548 INFO:teuthology.orchestra.run.smithi148.stdout:(7/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 112 kB/s | 20 kB 00:00 2024-03-04T12:13:15.564 INFO:teuthology.orchestra.run.smithi148.stdout:(8/45): libcap-devel-2.48-9.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-03-04T12:13:15.590 INFO:teuthology.orchestra.run.smithi148.stdout:(9/45): libnbd-1.18.1-3.el9.x86_64.rpm 3.9 MB/s | 165 kB 00:00 2024-03-04T12:13:15.615 INFO:teuthology.orchestra.run.smithi148.stdout:(10/45): sqlite-3.34.1-7.el9.x86_64.rpm 11 MB/s | 747 kB 00:00 2024-03-04T12:13:15.640 INFO:teuthology.orchestra.run.smithi148.stdout:(11/45): udftools-2.2-5.el9.x86_64.rpm 1.9 MB/s | 144 kB 00:00 2024-03-04T12:13:15.657 INFO:teuthology.orchestra.run.smithi148.stdout:(12/45): rmt-1.6-6.el9.x86_64.rpm 687 kB/s | 45 kB 00:00 2024-03-04T12:13:15.824 INFO:teuthology.orchestra.run.smithi148.stdout:(13/45): gdbm-devel-1.19-4.el9.x86_64.rpm 352 kB/s | 59 kB 00:00 2024-03-04T12:13:15.899 INFO:teuthology.orchestra.run.smithi148.stdout:(14/45): gdbm-1.19-4.el9.x86_64.rpm 511 kB/s | 132 kB 00:00 2024-03-04T12:13:16.334 INFO:teuthology.orchestra.run.smithi148.stdout:(15/45): dump-0.4-0.51.b47.el9.x86_64.rpm 173 kB/s | 88 kB 00:00 2024-03-04T12:13:16.509 INFO:teuthology.orchestra.run.smithi148.stdout:(16/45): indent-2.2.13-5.el9.x86_64.rpm 295 kB/s | 179 kB 00:00 2024-03-04T12:13:16.717 INFO:teuthology.orchestra.run.smithi148.stdout:(17/45): cpp-11.4.1-3.el9.x86_64.rpm 28 MB/s | 11 MB 00:00 2024-03-04T12:13:17.484 INFO:teuthology.orchestra.run.smithi148.stdout:(18/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 17 MB/s | 13 MB 00:00 2024-03-04T12:13:17.843 INFO:teuthology.orchestra.run.smithi148.stdout:(19/45): gcc-11.4.1-3.el9.x86_64.rpm 24 MB/s | 32 MB 00:01 2024-03-04T12:13:17.868 INFO:teuthology.orchestra.run.smithi148.stdout:(20/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 121 kB/s | 46 kB 00:00 2024-03-04T12:13:17.893 INFO:teuthology.orchestra.run.smithi148.stdout:(21/45): git-2.43.0-1.el9.x86_64.rpm 994 kB/s | 50 kB 00:00 2024-03-04T12:13:18.035 INFO:teuthology.orchestra.run.smithi148.stdout:(22/45): git-core-2.43.0-1.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-03-04T12:13:18.102 INFO:teuthology.orchestra.run.smithi148.stdout:(23/45): git-core-doc-2.43.0-1.el9.noarch.rpm 13 MB/s | 2.6 MB 00:00 2024-03-04T12:13:18.153 INFO:teuthology.orchestra.run.smithi148.stdout:(24/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 19 MB/s | 2.2 MB 00:00 2024-03-04T12:13:18.178 INFO:teuthology.orchestra.run.smithi148.stdout:(25/45): perl-Git-2.43.0-1.el9.noarch.rpm 504 kB/s | 37 kB 00:00 2024-03-04T12:13:18.203 INFO:teuthology.orchestra.run.smithi148.stdout:(26/45): python-unversioned-command-3.9.18-3.el 166 kB/s | 8.3 kB 00:00 2024-03-04T12:13:18.228 INFO:teuthology.orchestra.run.smithi148.stdout:(27/45): python3-devel-3.9.18-3.el9.x86_64.rpm 4.0 MB/s | 204 kB 00:00 2024-03-04T12:13:18.253 INFO:teuthology.orchestra.run.smithi148.stdout:(28/45): device-mapper-1.02.197-2.el9.x86_64.rp 2.7 MB/s | 139 kB 00:00 2024-03-04T12:13:18.279 INFO:teuthology.orchestra.run.smithi148.stdout:(29/45): device-mapper-event-1.02.197-2.el9.x86 660 kB/s | 33 kB 00:00 2024-03-04T12:13:18.304 INFO:teuthology.orchestra.run.smithi148.stdout:(30/45): device-mapper-event-libs-1.02.197-2.el 618 kB/s | 31 kB 00:00 2024-03-04T12:13:18.338 INFO:teuthology.orchestra.run.smithi148.stdout:(31/45): device-mapper-libs-1.02.197-2.el9.x86_ 3.0 MB/s | 178 kB 00:00 2024-03-04T12:13:18.372 INFO:teuthology.orchestra.run.smithi148.stdout:(32/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-04T12:13:18.397 INFO:teuthology.orchestra.run.smithi148.stdout:(33/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-03-04T12:13:18.423 INFO:teuthology.orchestra.run.smithi148.stdout:(34/45): libcom_err-1.46.5-5.el9.x86_64.rpm 505 kB/s | 25 kB 00:00 2024-03-04T12:13:18.447 INFO:teuthology.orchestra.run.smithi148.stdout:(35/45): libgcc-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 91 kB 00:00 2024-03-04T12:13:18.473 INFO:teuthology.orchestra.run.smithi148.stdout:(36/45): libgomp-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 267 kB 00:00 2024-03-04T12:13:18.498 INFO:teuthology.orchestra.run.smithi148.stdout:(37/45): libss-1.46.5-5.el9.x86_64.rpm 620 kB/s | 31 kB 00:00 2024-03-04T12:13:18.531 INFO:teuthology.orchestra.run.smithi148.stdout:(38/45): libstdc++-11.4.1-3.el9.x86_64.rpm 12 MB/s | 739 kB 00:00 2024-03-04T12:13:18.582 INFO:teuthology.orchestra.run.smithi148.stdout:(39/45): lvm2-2.03.23-2.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-03-04T12:13:18.624 INFO:teuthology.orchestra.run.smithi148.stdout:(40/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-04T12:13:18.649 INFO:teuthology.orchestra.run.smithi148.stdout:(41/45): make-4.3-8.el9.x86_64.rpm 7.9 MB/s | 529 kB 00:00 2024-03-04T12:13:18.675 INFO:teuthology.orchestra.run.smithi148.stdout:(42/45): python3-3.9.18-3.el9.x86_64.rpm 505 kB/s | 25 kB 00:00 2024-03-04T12:13:19.042 INFO:teuthology.orchestra.run.smithi148.stdout:(43/45): python3-libs-3.9.18-3.el9.x86_64.rpm 19 MB/s | 7.3 MB 00:00 2024-03-04T12:13:19.067 INFO:teuthology.orchestra.run.smithi148.stdout:(44/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 1.5 MB/s | 619 kB 00:00 2024-03-04T12:13:20.268 INFO:teuthology.orchestra.run.smithi148.stdout:(45/45): kernel-devel-5.14.0-427.el9.x86_64.rpm 4.7 MB/s | 22 MB 00:04 2024-03-04T12:13:20.270 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-04T12:13:20.270 INFO:teuthology.orchestra.run.smithi148.stdout:Total 18 MB/s | 108 MB 00:06 2024-03-04T12:13:20.484 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-04T12:13:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-04T12:13:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-04T12:13:22.024 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-04T12:13:22.024 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-04T12:13:24.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:13:24.476 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:13:24.811 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-04T12:13:24.860 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-04T12:13:24.923 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-04T12:13:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-03-04T12:13:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-03-04T12:13:25.102 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-03-04T12:13:25.175 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-03-04T12:13:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-03-04T12:13:25.260 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-03-04T12:13:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 8/73 2024-03-04T12:13:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 9/73 2024-03-04T12:13:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 10/73 2024-03-04T12:13:26.744 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 11/73 2024-03-04T12:13:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : make-1:4.3-8.el9.x86_64 12/73 2024-03-04T12:13:27.950 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 13/73 2024-03-04T12:13:28.053 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-03-04T12:13:28.086 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-03-04T12:13:28.129 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-03-04T12:13:28.153 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-04T12:13:28.356 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-04T12:13:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-03-04T12:13:29.027 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-03-04T12:13:29.054 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-03-04T12:13:29.425 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 21/73 2024-03-04T12:13:30.391 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 22/73 2024-03-04T12:13:30.597 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-03-04T12:13:30.643 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-03-04T12:13:30.677 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-03-04T12:13:30.713 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rmt-2:1.6-6.el9.x86_64 25/73 2024-03-04T12:13:30.742 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 26/73 2024-03-04T12:13:30.768 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 27/73 2024-03-04T12:13:30.793 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 28/73 2024-03-04T12:13:30.818 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 29/73 2024-03-04T12:13:30.843 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 30/73 2024-03-04T12:13:30.869 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 31/73 2024-03-04T12:13:31.021 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 32/73 2024-03-04T12:13:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fio-3.35-1.el9.x86_64 33/73 2024-03-04T12:13:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-03-04T12:13:40.383 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-03-04T12:13:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-devel-5.14.0-427.el9.x86_64 36/73 2024-03-04T12:13:51.127 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-devel-5.14.0-427.el9.x86_64 36/73 2024-03-04T12:13:51.197 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-03-04T12:13:51.229 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-04T12:13:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-04T12:13:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-03-04T12:13:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-04T12:13:51.518 INFO:teuthology.orchestra.run.smithi148.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-03-04T12:13:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:51.810 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-04T12:13:51.869 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-03-04T12:13:51.920 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-03-04T12:13:51.975 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-03-04T12:13:52.015 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-03-04T12:13:52.057 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-03-04T12:13:52.084 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-03-04T12:13:52.117 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 47/73 2024-03-04T12:13:52.144 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 48/73 2024-03-04T12:13:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 49/73 2024-03-04T12:13:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-03-04T12:13:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-03-04T12:13:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-03-04T12:13:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-03-04T12:13:52.294 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-03-04T12:13:52.319 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-03-04T12:13:52.324 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 54/73 2024-03-04T12:13:52.353 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 54/73 2024-03-04T12:13:52.379 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 55/73 2024-03-04T12:13:52.379 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 56/73 2024-03-04T12:13:52.440 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 56/73 2024-03-04T12:13:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 57/73 2024-03-04T12:13:52.530 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 58/73 2024-03-04T12:13:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 59/73 2024-03-04T12:13:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 60/73 2024-03-04T12:13:52.623 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 61/73 2024-03-04T12:13:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-03-04T12:13:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-03-04T12:13:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-03-04T12:13:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-03-04T12:13:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-03-04T12:13:52.843 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-03-04T12:13:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-03-04T12:13:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-04T12:13:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-04T12:13:52.979 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 70/73 2024-03-04T12:13:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 71/73 2024-03-04T12:13:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : make-1:4.3-7.el9.x86_64 72/73 2024-03-04T12:13:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 73/73 2024-03-04T12:13:54.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:13:54.540 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:13:55.642 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: git-core-2.39.3-1.el9.x86_64 73/73 2024-03-04T12:13:55.642 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 8/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 9/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 10/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : udftools-2.2-5.el9.x86_64 11/73 2024-03-04T12:13:55.643 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 12/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-devel-5.14.0-427.el9.x86_64 13/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 18/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 19/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 20/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 21/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 22/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 23/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 24/73 2024-03-04T12:13:55.644 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 25/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-2.43.0-1.el9.x86_64 26/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-2.39.3-1.el9.x86_64 27/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 28/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 29/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 30/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 31/73 2024-03-04T12:13:55.649 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 32/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 33/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 34/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 35/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 36/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 37/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 38/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 39/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 40/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 41/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 42/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 43/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 44/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 45/73 2024-03-04T12:13:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 46/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 47/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 48/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 49/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 50/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 51/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 52/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 53/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 54/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 55/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 56/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 57/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libss-1.46.5-5.el9.x86_64 58/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libss-1.46.5-3.el9.x86_64 59/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 60/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 61/73 2024-03-04T12:13:55.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 62/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 63/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 64/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 65/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : make-1:4.3-8.el9.x86_64 66/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : make-1:4.3-7.el9.x86_64 67/73 2024-03-04T12:13:55.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-3.9.18-3.el9.x86_64 68/73 2024-03-04T12:13:55.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-3.9.17-2.el9.x86_64 69/73 2024-03-04T12:13:55.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 70/73 2024-03-04T12:13:55.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 71/73 2024-03-04T12:13:55.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 72/73 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 73/73 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: cpp-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-03-04T12:13:56.883 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: gcc-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: git-2.43.0-1.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: git-core-2.43.0-1.el9.x86_64 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-03-04T12:13:56.884 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: libgcc-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: libgomp-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: libss-1.46.5-5.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: make-1:4.3-8.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: perl-Git-2.43.0-1.el9.noarch 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: python3-3.9.18-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-03-04T12:13:56.885 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-427.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: udftools-2.2-5.el9.x86_64 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:56.886 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-04T12:13:57.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo useradd -m fsgqa 2024-03-04T12:13:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:useradd: user 'fsgqa' already exists 2024-03-04T12:13:57.692 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-04T12:13:57.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo groupadd fsgqa 2024-03-04T12:13:57.772 INFO:teuthology.orchestra.run.smithi148.stderr:groupadd: group 'fsgqa' already exists 2024-03-04T12:13:57.774 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-04T12:13:57.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo useradd fsgqa2 2024-03-04T12:13:58.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo useradd 123456-fsgqa 2024-03-04T12:13:58.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-03-04T12:13:59.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.050+0000 7f546e259640 1 Processor -- start 2024-03-04T12:13:59.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.050+0000 7f546e259640 1 -- start start 2024-03-04T12:13:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f546e259640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54680cfd60 0x7f54680cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f546e259640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cf0d0 0x7f54680cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f546e259640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54680d0140 con 0x7f546806d3b0 2024-03-04T12:13:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f546e259640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546815cbb0 con 0x7f54680cfd60 2024-03-04T12:13:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f546e259640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546815c280 con 0x7f54680cf0d0 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.051+0000 7f54677fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54680cfd60 0x7f54680cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54680cfd60 0x7f54680cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39846/0 (socket says 172.21.15.148:39846) 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 -- 172.21.15.148:0/3406621233 learned_addr learned my addr 172.21.15.148:0/3406621233 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f5467fff640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cf0d0 0x7f54680cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 -- 172.21.15.148:0/3406621233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cf0d0 msgr2=0x7f54680cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cf0d0 0x7f54680cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 -- 172.21.15.148:0/3406621233 >> v1:172.21.15.191:6789/0 conn(0x7f546806d3b0 legacy=0x7f54680ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.052+0000 7f54677fe640 1 -- 172.21.15.148:0/3406621233 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546815c560 con 0x7f54680cfd60 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f54677fe640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cfd60 0x7f54680cbdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5458009870 tx=0x7f545802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70abbca6735b0fb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f54667fc640 1 -- 172.21.15.148:0/3406621233 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458035020 con 0x7f54680cfd60 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f54667fc640 1 -- 172.21.15.148:0/3406621233 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5458031640 con 0x7f54680cfd60 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f54667fc640 1 -- 172.21.15.148:0/3406621233 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545802d040 con 0x7f54680cfd60 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 -- 172.21.15.148:0/3406621233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cfd60 msgr2=0x7f54680cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cfd60 0x7f54680cbdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5458009870 tx=0x7f545802b8f0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 -- 172.21.15.148:0/3406621233 shutdown_connections 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cf0d0 0x7f54680cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 --2- 172.21.15.148:0/3406621233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cfd60 0x7f54680cbdf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 -- 172.21.15.148:0/3406621233 >> 172.21.15.148:0/3406621233 conn(0x7f546806acd0 msgr2=0x7f546806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 -- 172.21.15.148:0/3406621233 shutdown_connections 2024-03-04T12:13:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.053+0000 7f546e259640 1 -- 172.21.15.148:0/3406621233 wait complete. 2024-03-04T12:13:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 Processor -- start 2024-03-04T12:13:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 -- start start 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f546806d3b0 0x7f546814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cfd60 0x7f546814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546815f520 con 0x7f54680cf0d0 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546815bf30 con 0x7f546806d3b0 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.055+0000 7f546e259640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546815f6a0 con 0x7f54680cfd60 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39856/0 (socket says 172.21.15.148:39856) 2024-03-04T12:13:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 -- 172.21.15.148:0/146392036 learned_addr learned my addr 172.21.15.148:0/146392036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f54677fe640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f546806d3b0 0x7f546814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5467fff640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cfd60 0x7f546814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 -- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cfd60 msgr2=0x7f546814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cfd60 0x7f546814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 -- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f546806d3b0 msgr2=0x7f546814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f546806d3b0 0x7f546814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 -- 172.21.15.148:0/146392036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680b2250 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f5466ffd640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c470 tx=0x7f545c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0052d0 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c005470 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54681f1720 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c007260 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.056+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681f1ca0 con 0x7f54680cf0d0 2024-03-04T12:13:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.057+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468074f60 con 0x7f54680cf0d0 2024-03-04T12:13:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.058+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f545c007400 con 0x7f54680cf0d0 2024-03-04T12:13:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.058+0000 7f544ffff640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 0x7f545406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.058+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f545c089780 con 0x7f54680cf0d0 2024-03-04T12:13:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.058+0000 7f54677fe640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 0x7f545406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.058+0000 7f54677fe640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 0x7f545406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5458036000 tx=0x7f545803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.062+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f545c057e90 con 0x7f54680cf0d0 2024-03-04T12:13:59.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.297+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5468154f70 con 0x7f54680cf0d0 2024-03-04T12:13:59.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.298+0000 7f544ffff640 1 -- 172.21.15.148:0/146392036 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f545c05bf30 con 0x7f54680cf0d0 2024-03-04T12:13:59.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:13:59.301 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","modified":"2024-03-04T12:10:55.960578Z","created":"2024-03-04T12:10:55.960578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3301","nonce":0},{"type":"v1","addr":"172.21.15.191:6790","nonce":0}]},"addr":"172.21.15.191:6790/0","public_addr":"172.21.15.191:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-04T12:13:59.301 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-04T12:13:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 msgr2=0x7f545406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 0x7f545406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5458036000 tx=0x7f545803c040 comp rx=0 tx=0).stop 2024-03-04T12:13:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 msgr2=0x7f546814dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c470 tx=0x7f545c001e70 comp rx=0 tx=0).stop 2024-03-04T12:13:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 shutdown_connections 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f545406b4d0 0x7f545406d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54680cfd60 0x7f546814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680cf0d0 0x7f546814dfd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 --2- 172.21.15.148:0/146392036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f546806d3b0 0x7f546814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 >> 172.21.15.148:0/146392036 conn(0x7f546806acd0 msgr2=0x7f546806b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 shutdown_connections 2024-03-04T12:13:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.300+0000 7f546e259640 1 -- 172.21.15.148:0/146392036 wait complete. 2024-03-04T12:13:59.314 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-03-04T12:13:59.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.412+0000 7fe6b8236640 1 Processor -- start 2024-03-04T12:13:59.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.412+0000 7fe6b8236640 1 -- start start 2024-03-04T12:13:59.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.413+0000 7fe6b8236640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.413+0000 7fe6b8236640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe6b00cf0d0 0x7fe6b00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.413+0000 7fe6b8236640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b00d0140 con 0x7fe6b006d3b0 2024-03-04T12:13:59.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.413+0000 7fe6b8236640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b015cbb0 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.413+0000 7fe6b8236640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b015c280 con 0x7fe6b00cfd60 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b67d3640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe6b00cf0d0 0x7fe6b00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:60470/0 (socket says 172.21.15.148:60470) 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 -- 172.21.15.148:0/143126320 learned_addr learned my addr 172.21.15.148:0/143126320 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b57d1640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fe6b006d3b0 0x7fe6b00ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:47308/0 (socket says 172.21.15.148:47308) 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 -- 172.21.15.148:0/143126320 >> v2:172.21.15.191:3300/0 conn(0x7fe6b00cf0d0 msgr2=0x7fe6b00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 --2- 172.21.15.148:0/143126320 >> v2:172.21.15.191:3300/0 conn(0x7fe6b00cf0d0 0x7fe6b00cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 -- 172.21.15.148:0/143126320 >> v1:172.21.15.191:6789/0 conn(0x7fe6b006d3b0 legacy=0x7fe6b00ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.414+0000 7fe6b5fd2640 1 -- 172.21.15.148:0/143126320 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b015c560 con 0x7fe6b00cfd60 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b5fd2640 1 --2- 172.21.15.148:0/143126320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b015cd30 tx=0x7fe6a802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ffcafd96d087a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b4fd0640 1 -- 172.21.15.148:0/143126320 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8034070 con 0x7fe6b00cfd60 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b4fd0640 1 -- 172.21.15.148:0/143126320 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8030840 con 0x7fe6b00cfd60 2024-03-04T12:13:59.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b4fd0640 1 -- 172.21.15.148:0/143126320 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8030c10 con 0x7fe6b00cfd60 2024-03-04T12:13:59.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b8236640 1 -- 172.21.15.148:0/143126320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 msgr2=0x7fe6b00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.415+0000 7fe6b8236640 1 --2- 172.21.15.148:0/143126320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b015cd30 tx=0x7fe6a802b980 comp rx=0 tx=0).stop 2024-03-04T12:13:59.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 -- 172.21.15.148:0/143126320 shutdown_connections 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 --2- 172.21.15.148:0/143126320 >> v2:172.21.15.191:3300/0 conn(0x7fe6b00cf0d0 0x7fe6b00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 --2- 172.21.15.148:0/143126320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b00cfd60 0x7fe6b00cbdf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 -- 172.21.15.148:0/143126320 >> 172.21.15.148:0/143126320 conn(0x7fe6b006acd0 msgr2=0x7fe6b006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 -- 172.21.15.148:0/143126320 shutdown_connections 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.416+0000 7fe6b8236640 1 -- 172.21.15.148:0/143126320 wait complete. 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 Processor -- start 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 -- start start 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b006d3b0 0x7fe6b014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b00cfd60 0x7fe6b0152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b015f520 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b015bf30 con 0x7fe6b00cfd60 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.417+0000 7fe6b8236640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b015f6a0 con 0x7fe6b006d3b0 2024-03-04T12:13:59.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39864/0 (socket says 172.21.15.148:39864) 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 -- 172.21.15.148:0/1201978635 learned_addr learned my addr 172.21.15.148:0/1201978635 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b5fd2640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b006d3b0 0x7fe6b014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b67d3640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b00cfd60 0x7fe6b0152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 -- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b006d3b0 msgr2=0x7fe6b014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b006d3b0 0x7fe6b014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 -- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b00cfd60 msgr2=0x7fe6b0152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b00cfd60 0x7fe6b0152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 -- 172.21.15.148:0/1201978635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0157b90 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b57d1640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c440 tx=0x7fe6a0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00074e0 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0007680 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b0157e70 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004a20 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.418+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b0158430 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.419+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0158630 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.420+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6a001a020 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.420+0000 7fe6a67fc640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T12:13:59.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.420+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe6a00899a0 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.420+0000 7fe6b5fd2640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T12:13:59.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.421+0000 7fe6b5fd2640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8036000 tx=0x7fe6a803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T12:13:59.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.424+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6b0158630 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.658+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7fe6b0074f60 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.658+0000 7fe6a67fc640 1 -- 172.21.15.148:0/1201978635 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v13) v1 ==== 92+0+63 (secure 0 0 0) 0x7fe6a00580b0 con 0x7fe6b00cf0d0 2024-03-04T12:13:59.661 INFO:teuthology.orchestra.run.smithi148.stdout:[client.admin] 2024-03-04T12:13:59.661 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.660+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 msgr2=0x7fe69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.660+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8036000 tx=0x7fe6a803b040 comp rx=0 tx=0).stop 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 msgr2=0x7fe6b01500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c440 tx=0x7fe6a0001e70 comp rx=0 tx=0).stop 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 shutdown_connections 2024-03-04T12:13:59.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b00cfd60 0x7fe6b0152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b00cf0d0 0x7fe6b01500b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 --2- 172.21.15.148:0/1201978635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b006d3b0 0x7fe6b014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 >> 172.21.15.148:0/1201978635 conn(0x7fe6b006acd0 msgr2=0x7fe6b006e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 shutdown_connections 2024-03-04T12:13:59.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T12:13:59.661+0000 7fe6b8236640 1 -- 172.21.15.148:0/1201978635 wait complete. 2024-03-04T12:13:59.675 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:13:59.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/tmp/tmp.ATjWjXU3R3xfstests-dev/local.config 2024-03-04T12:13:59.701 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.148:6789:/test export TEST_DIR=/tmp/tmp.1jHsm6tBGxtest export SCRATCH_DEV=172.21.15.148:6789:/scratch export SCRATCH_MNT=/tmp/tmp.PKHzEB1Z0hscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w==" 2024-03-04T12:13:59.702 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T12:13:59.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/tmp/tmp.ATjWjXU3R3xfstests-dev/ceph.exclude 2024-03-04T12:13:59.769 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo make) 2024-03-04T12:13:59.839 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize -cfi 2024-03-04T12:14:00.192 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-04T12:14:00.192 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file './config.guess' 2024-03-04T12:14:00.195 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file './config.sub' 2024-03-04T12:14:00.198 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file './install-sh' 2024-03-04T12:14:00.202 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file './ltmain.sh' 2024-03-04T12:14:00.224 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-04T12:14:00.224 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-04T12:14:00.228 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-04T12:14:00.232 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-04T12:14:00.237 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-04T12:14:00.241 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-04T12:14:00.248 INFO:teuthology.orchestra.run.smithi148.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-04T12:14:00.249 INFO:teuthology.orchestra.run.smithi148.stdout:cp include/install-sh . 2024-03-04T12:14:00.249 INFO:teuthology.orchestra.run.smithi148.stdout:aclocal -I m4 2024-03-04T12:14:01.258 INFO:teuthology.orchestra.run.smithi148.stdout:autoheader 2024-03-04T12:14:01.544 INFO:teuthology.orchestra.run.smithi148.stdout:autoconf 2024-03-04T12:14:01.711 INFO:teuthology.orchestra.run.smithi148.stdout:./configure \ 2024-03-04T12:14:01.711 INFO:teuthology.orchestra.run.smithi148.stdout: --libexecdir=/usr/lib \ 2024-03-04T12:14:01.712 INFO:teuthology.orchestra.run.smithi148.stdout: --exec_prefix=/var/lib 2024-03-04T12:14:01.914 INFO:teuthology.orchestra.run.smithi148.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-04T12:14:01.914 INFO:teuthology.orchestra.run.smithi148.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-04T12:14:01.915 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to print strings... printf 2024-03-04T12:14:01.916 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gcc... gcc 2024-03-04T12:14:01.969 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether the C compiler works... yes 2024-03-04T12:14:01.969 INFO:teuthology.orchestra.run.smithi148.stdout:checking for C compiler default output file name... a.out 2024-03-04T12:14:01.995 INFO:teuthology.orchestra.run.smithi148.stdout:checking for suffix of executables... 2024-03-04T12:14:02.047 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether we are cross compiling... no 2024-03-04T12:14:02.064 INFO:teuthology.orchestra.run.smithi148.stdout:checking for suffix of object files... o 2024-03-04T12:14:02.078 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether we are using the GNU C compiler... yes 2024-03-04T12:14:02.093 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether gcc accepts -g... yes 2024-03-04T12:14:02.120 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-04T12:14:02.122 INFO:teuthology.orchestra.run.smithi148.stdout:checking for a sed that does not truncate output... /bin/sed 2024-03-04T12:14:02.125 INFO:teuthology.orchestra.run.smithi148.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-03-04T12:14:02.126 INFO:teuthology.orchestra.run.smithi148.stdout:checking for egrep... /bin/grep -E 2024-03-04T12:14:02.127 INFO:teuthology.orchestra.run.smithi148.stdout:checking for fgrep... /bin/grep -F 2024-03-04T12:14:02.130 INFO:teuthology.orchestra.run.smithi148.stdout:checking for ld used by gcc... /bin/ld 2024-03-04T12:14:02.131 INFO:teuthology.orchestra.run.smithi148.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-03-04T12:14:02.137 INFO:teuthology.orchestra.run.smithi148.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-03-04T12:14:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-03-04T12:14:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether ln -s works... yes 2024-03-04T12:14:03.100 INFO:teuthology.orchestra.run.smithi148.stdout:checking the maximum length of command line arguments... 1572864 2024-03-04T12:14:03.104 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking for /bin/ld option to reload object files... -r 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking for objdump... objdump 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to recognize dependent libraries... pass_all 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dlltool... no 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-04T12:14:03.105 INFO:teuthology.orchestra.run.smithi148.stdout:checking for ar... ar 2024-03-04T12:14:03.150 INFO:teuthology.orchestra.run.smithi148.stdout:checking for archiver @FILE support... @ 2024-03-04T12:14:03.150 INFO:teuthology.orchestra.run.smithi148.stdout:checking for strip... strip 2024-03-04T12:14:03.150 INFO:teuthology.orchestra.run.smithi148.stdout:checking for ranlib... ranlib 2024-03-04T12:14:03.150 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gawk... gawk 2024-03-04T12:14:03.215 INFO:teuthology.orchestra.run.smithi148.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-03-04T12:14:03.216 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sysroot... no 2024-03-04T12:14:03.219 INFO:teuthology.orchestra.run.smithi148.stdout:checking for a working dd... /bin/dd 2024-03-04T12:14:03.221 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-03-04T12:14:03.234 INFO:teuthology.orchestra.run.smithi148.stdout:checking for mt... no 2024-03-04T12:14:03.236 INFO:teuthology.orchestra.run.smithi148.stdout:checking if : is a manifest tool... no 2024-03-04T12:14:03.277 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to run the C preprocessor... gcc -E 2024-03-04T12:14:03.447 INFO:teuthology.orchestra.run.smithi148.stdout:checking for ANSI C header files... yes 2024-03-04T12:14:03.467 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/types.h... yes 2024-03-04T12:14:03.493 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/stat.h... yes 2024-03-04T12:14:03.516 INFO:teuthology.orchestra.run.smithi148.stdout:checking for stdlib.h... yes 2024-03-04T12:14:03.540 INFO:teuthology.orchestra.run.smithi148.stdout:checking for string.h... yes 2024-03-04T12:14:03.568 INFO:teuthology.orchestra.run.smithi148.stdout:checking for memory.h... yes 2024-03-04T12:14:03.591 INFO:teuthology.orchestra.run.smithi148.stdout:checking for strings.h... yes 2024-03-04T12:14:03.625 INFO:teuthology.orchestra.run.smithi148.stdout:checking for inttypes.h... yes 2024-03-04T12:14:03.650 INFO:teuthology.orchestra.run.smithi148.stdout:checking for stdint.h... yes 2024-03-04T12:14:03.688 INFO:teuthology.orchestra.run.smithi148.stdout:checking for unistd.h... yes 2024-03-04T12:14:03.718 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dlfcn.h... yes 2024-03-04T12:14:03.721 INFO:teuthology.orchestra.run.smithi148.stdout:checking for objdir... .libs 2024-03-04T12:14:03.784 INFO:teuthology.orchestra.run.smithi148.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-04T12:14:03.784 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-04T12:14:03.798 INFO:teuthology.orchestra.run.smithi148.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-04T12:14:03.815 INFO:teuthology.orchestra.run.smithi148.stdout:checking if gcc static flag -static works... no 2024-03-04T12:14:03.836 INFO:teuthology.orchestra.run.smithi148.stdout:checking if gcc supports -c -o file.o... yes 2024-03-04T12:14:03.836 INFO:teuthology.orchestra.run.smithi148.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-04T12:14:03.842 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-04T12:14:03.867 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether -lc should be explicitly linked in... no 2024-03-04T12:14:03.923 INFO:teuthology.orchestra.run.smithi148.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-04T12:14:03.924 INFO:teuthology.orchestra.run.smithi148.stdout:checking how to hardcode library paths into programs... immediate 2024-03-04T12:14:03.925 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether stripping libraries is possible... yes 2024-03-04T12:14:03.925 INFO:teuthology.orchestra.run.smithi148.stdout:checking if libtool supports shared libraries... yes 2024-03-04T12:14:03.925 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether to build shared libraries... yes 2024-03-04T12:14:03.925 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether to build static libraries... yes 2024-03-04T12:14:03.931 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gcc... (cached) gcc 2024-03-04T12:14:03.947 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-04T12:14:03.947 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-04T12:14:03.948 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-04T12:14:03.948 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gmake... /usr/bin/gmake 2024-03-04T12:14:03.948 INFO:teuthology.orchestra.run.smithi148.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-03-04T12:14:03.950 INFO:teuthology.orchestra.run.smithi148.stdout:checking for tar... /bin/tar 2024-03-04T12:14:03.950 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gzip... /bin/gzip 2024-03-04T12:14:03.956 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether gcc -MM is supported... yes 2024-03-04T12:14:03.957 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sort... /bin/sort 2024-03-04T12:14:03.957 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether ln -s works... yes 2024-03-04T12:14:03.957 INFO:teuthology.orchestra.run.smithi148.stdout:checking for rpm... /bin/rpm 2024-03-04T12:14:03.959 INFO:teuthology.orchestra.run.smithi148.stdout:checking for rpmbuild... no 2024-03-04T12:14:03.985 INFO:teuthology.orchestra.run.smithi148.stdout:checking assert.h usability... yes 2024-03-04T12:14:03.993 INFO:teuthology.orchestra.run.smithi148.stdout:checking assert.h presence... yes 2024-03-04T12:14:03.993 INFO:teuthology.orchestra.run.smithi148.stdout:checking for assert.h... yes 2024-03-04T12:14:04.020 INFO:teuthology.orchestra.run.smithi148.stdout:checking bstring.h usability... no 2024-03-04T12:14:04.031 INFO:teuthology.orchestra.run.smithi148.stdout:checking bstring.h presence... no 2024-03-04T12:14:04.031 INFO:teuthology.orchestra.run.smithi148.stdout:checking for bstring.h... no 2024-03-04T12:14:04.055 INFO:teuthology.orchestra.run.smithi148.stdout:checking libgen.h usability... yes 2024-03-04T12:14:04.063 INFO:teuthology.orchestra.run.smithi148.stdout:checking libgen.h presence... yes 2024-03-04T12:14:04.063 INFO:teuthology.orchestra.run.smithi148.stdout:checking for libgen.h... yes 2024-03-04T12:14:04.100 INFO:teuthology.orchestra.run.smithi148.stdout:checking dirent.h usability... yes 2024-03-04T12:14:04.109 INFO:teuthology.orchestra.run.smithi148.stdout:checking dirent.h presence... yes 2024-03-04T12:14:04.109 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dirent.h... yes 2024-03-04T12:14:04.159 INFO:teuthology.orchestra.run.smithi148.stdout:checking errno.h usability... yes 2024-03-04T12:14:04.167 INFO:teuthology.orchestra.run.smithi148.stdout:checking errno.h presence... yes 2024-03-04T12:14:04.167 INFO:teuthology.orchestra.run.smithi148.stdout:checking for errno.h... yes 2024-03-04T12:14:04.198 INFO:teuthology.orchestra.run.smithi148.stdout:checking malloc.h usability... yes 2024-03-04T12:14:04.208 INFO:teuthology.orchestra.run.smithi148.stdout:checking malloc.h presence... yes 2024-03-04T12:14:04.208 INFO:teuthology.orchestra.run.smithi148.stdout:checking for malloc.h... yes 2024-03-04T12:14:04.234 INFO:teuthology.orchestra.run.smithi148.stdout:checking uuid.h usability... no 2024-03-04T12:14:04.244 INFO:teuthology.orchestra.run.smithi148.stdout:checking uuid.h presence... no 2024-03-04T12:14:04.244 INFO:teuthology.orchestra.run.smithi148.stdout:checking for uuid.h... no 2024-03-04T12:14:04.285 INFO:teuthology.orchestra.run.smithi148.stdout:checking uuid/uuid.h usability... yes 2024-03-04T12:14:04.295 INFO:teuthology.orchestra.run.smithi148.stdout:checking uuid/uuid.h presence... yes 2024-03-04T12:14:04.296 INFO:teuthology.orchestra.run.smithi148.stdout:checking for uuid/uuid.h... yes 2024-03-04T12:14:04.321 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/uuid.h usability... no 2024-03-04T12:14:04.332 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/uuid.h presence... no 2024-03-04T12:14:04.332 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/uuid.h... no 2024-03-04T12:14:04.358 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/file.h usability... yes 2024-03-04T12:14:04.366 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/file.h presence... yes 2024-03-04T12:14:04.367 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/file.h... yes 2024-03-04T12:14:04.403 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fcntl.h usability... yes 2024-03-04T12:14:04.412 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fcntl.h presence... yes 2024-03-04T12:14:04.412 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/fcntl.h... yes 2024-03-04T12:14:04.439 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/syssgi.h usability... no 2024-03-04T12:14:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/syssgi.h presence... no 2024-03-04T12:14:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/syssgi.h... no 2024-03-04T12:14:04.491 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/param.h usability... yes 2024-03-04T12:14:04.502 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/param.h presence... yes 2024-03-04T12:14:04.502 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/param.h... yes 2024-03-04T12:14:04.506 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/stat.h... (cached) yes 2024-03-04T12:14:04.536 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/statvfs.h usability... yes 2024-03-04T12:14:04.545 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/statvfs.h presence... yes 2024-03-04T12:14:04.545 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/statvfs.h... yes 2024-03-04T12:14:04.571 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/time.h usability... yes 2024-03-04T12:14:04.580 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/time.h presence... yes 2024-03-04T12:14:04.580 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/time.h... yes 2024-03-04T12:14:04.620 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/ioctl.h usability... yes 2024-03-04T12:14:04.628 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/ioctl.h presence... yes 2024-03-04T12:14:04.628 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/ioctl.h... yes 2024-03-04T12:14:04.662 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/wait.h usability... yes 2024-03-04T12:14:04.672 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/wait.h presence... yes 2024-03-04T12:14:04.672 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/wait.h... yes 2024-03-04T12:14:04.677 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/types.h... (cached) yes 2024-03-04T12:14:04.684 INFO:teuthology.orchestra.run.smithi148.stdout:checking for strings.h... (cached) yes 2024-03-04T12:14:04.710 INFO:teuthology.orchestra.run.smithi148.stdout:checking err.h usability... yes 2024-03-04T12:14:04.719 INFO:teuthology.orchestra.run.smithi148.stdout:checking err.h presence... yes 2024-03-04T12:14:04.719 INFO:teuthology.orchestra.run.smithi148.stdout:checking for err.h... yes 2024-03-04T12:14:04.747 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/falloc.h usability... yes 2024-03-04T12:14:04.754 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/falloc.h presence... yes 2024-03-04T12:14:04.755 INFO:teuthology.orchestra.run.smithi148.stdout:checking for linux/falloc.h... yes 2024-03-04T12:14:04.780 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-04T12:14:04.791 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-04T12:14:04.791 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-04T12:14:04.815 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-04T12:14:04.826 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-04T12:14:04.826 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-04T12:14:04.849 INFO:teuthology.orchestra.run.smithi148.stdout:checking xfs/platform_defs.h usability... no 2024-03-04T12:14:04.860 INFO:teuthology.orchestra.run.smithi148.stdout:checking xfs/platform_defs.h presence... no 2024-03-04T12:14:04.860 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/platform_defs.h... no 2024-03-04T12:14:04.884 INFO:teuthology.orchestra.run.smithi148.stdout:checking btrfs/ioctl.h usability... no 2024-03-04T12:14:04.894 INFO:teuthology.orchestra.run.smithi148.stdout:checking btrfs/ioctl.h presence... no 2024-03-04T12:14:04.894 INFO:teuthology.orchestra.run.smithi148.stdout:checking for btrfs/ioctl.h... no 2024-03-04T12:14:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:checking cifs/ioctl.h usability... no 2024-03-04T12:14:04.929 INFO:teuthology.orchestra.run.smithi148.stdout:checking cifs/ioctl.h presence... no 2024-03-04T12:14:04.929 INFO:teuthology.orchestra.run.smithi148.stdout:checking for cifs/ioctl.h... no 2024-03-04T12:14:04.955 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/mman.h usability... yes 2024-03-04T12:14:04.964 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/mman.h presence... yes 2024-03-04T12:14:04.964 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/mman.h... yes 2024-03-04T12:14:04.979 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/xfs_log_format.h... no 2024-03-04T12:14:05.013 INFO:teuthology.orchestra.run.smithi148.stdout:checking for uuid_compare... no 2024-03-04T12:14:05.043 INFO:teuthology.orchestra.run.smithi148.stdout:checking for uuid_compare in -luuid... yes 2024-03-04T12:14:05.115 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/xfs.h... yes 2024-03-04T12:14:05.127 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/libxfs.h... no 2024-03-04T12:14:05.142 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-04T12:14:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/xqm.h... yes 2024-03-04T12:14:05.219 INFO:teuthology.orchestra.run.smithi148.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-04T12:14:05.241 INFO:teuthology.orchestra.run.smithi148.stdout:checking xfs/handle.h usability... yes 2024-03-04T12:14:05.249 INFO:teuthology.orchestra.run.smithi148.stdout:checking xfs/handle.h presence... yes 2024-03-04T12:14:05.249 INFO:teuthology.orchestra.run.smithi148.stdout:checking for xfs/handle.h... yes 2024-03-04T12:14:05.281 INFO:teuthology.orchestra.run.smithi148.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-04T12:14:05.304 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/acl.h usability... yes 2024-03-04T12:14:05.314 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/acl.h presence... yes 2024-03-04T12:14:05.314 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/acl.h... yes 2024-03-04T12:14:05.337 INFO:teuthology.orchestra.run.smithi148.stdout:checking acl/libacl.h usability... yes 2024-03-04T12:14:05.346 INFO:teuthology.orchestra.run.smithi148.stdout:checking acl/libacl.h presence... yes 2024-03-04T12:14:05.346 INFO:teuthology.orchestra.run.smithi148.stdout:checking for acl/libacl.h... yes 2024-03-04T12:14:05.377 INFO:teuthology.orchestra.run.smithi148.stdout:checking for acl_init in -lacl... yes 2024-03-04T12:14:05.401 INFO:teuthology.orchestra.run.smithi148.stdout:checking gdbm-ndbm.h usability... no 2024-03-04T12:14:05.411 INFO:teuthology.orchestra.run.smithi148.stdout:checking gdbm-ndbm.h presence... no 2024-03-04T12:14:05.411 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gdbm-ndbm.h... no 2024-03-04T12:14:05.433 INFO:teuthology.orchestra.run.smithi148.stdout:checking gdbm/ndbm.h usability... yes 2024-03-04T12:14:05.443 INFO:teuthology.orchestra.run.smithi148.stdout:checking gdbm/ndbm.h presence... yes 2024-03-04T12:14:05.444 INFO:teuthology.orchestra.run.smithi148.stdout:checking for gdbm/ndbm.h... yes 2024-03-04T12:14:05.477 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_open in -lgdbm... no 2024-03-04T12:14:05.511 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_fetch in -lgdbm... no 2024-03-04T12:14:05.545 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_store in -lgdbm... no 2024-03-04T12:14:05.580 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_close in -lgdbm... no 2024-03-04T12:14:05.617 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-03-04T12:14:05.648 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-03-04T12:14:05.680 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-03-04T12:14:05.712 INFO:teuthology.orchestra.run.smithi148.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-03-04T12:14:05.735 INFO:teuthology.orchestra.run.smithi148.stdout:checking libaio.h usability... yes 2024-03-04T12:14:05.746 INFO:teuthology.orchestra.run.smithi148.stdout:checking libaio.h presence... yes 2024-03-04T12:14:05.746 INFO:teuthology.orchestra.run.smithi148.stdout:checking for libaio.h... yes 2024-03-04T12:14:05.768 INFO:teuthology.orchestra.run.smithi148.stdout:checking liburing.h usability... no 2024-03-04T12:14:05.779 INFO:teuthology.orchestra.run.smithi148.stdout:checking liburing.h presence... no 2024-03-04T12:14:05.779 INFO:teuthology.orchestra.run.smithi148.stdout:checking for liburing.h... no 2024-03-04T12:14:05.813 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/capability.h usability... yes 2024-03-04T12:14:05.823 INFO:teuthology.orchestra.run.smithi148.stdout:checking sys/capability.h presence... yes 2024-03-04T12:14:05.823 INFO:teuthology.orchestra.run.smithi148.stdout:checking for sys/capability.h... yes 2024-03-04T12:14:05.847 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/fiemap.h usability... yes 2024-03-04T12:14:05.854 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/fiemap.h presence... yes 2024-03-04T12:14:05.854 INFO:teuthology.orchestra.run.smithi148.stdout:checking for linux/fiemap.h... yes 2024-03-04T12:14:05.888 INFO:teuthology.orchestra.run.smithi148.stdout:checking for fallocate... yes 2024-03-04T12:14:05.921 INFO:teuthology.orchestra.run.smithi148.stdout:checking for open_by_handle_at... yes 2024-03-04T12:14:05.944 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/fs.h usability... yes 2024-03-04T12:14:05.953 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/fs.h presence... yes 2024-03-04T12:14:05.953 INFO:teuthology.orchestra.run.smithi148.stdout:checking for linux/fs.h... yes 2024-03-04T12:14:05.975 INFO:teuthology.orchestra.run.smithi148.stdout:checking btrfsutil.h usability... no 2024-03-04T12:14:05.985 INFO:teuthology.orchestra.run.smithi148.stdout:checking btrfsutil.h presence... no 2024-03-04T12:14:05.985 INFO:teuthology.orchestra.run.smithi148.stdout:checking for btrfsutil.h... no 2024-03-04T12:14:06.031 INFO:teuthology.orchestra.run.smithi148.stdout:checking for copy_file_range... yes 2024-03-04T12:14:06.066 INFO:teuthology.orchestra.run.smithi148.stdout:checking for SEEK_DATA... yes 2024-03-04T12:14:06.115 INFO:teuthology.orchestra.run.smithi148.stdout:checking for BMV_OF_SHARED... yes 2024-03-04T12:14:06.151 INFO:teuthology.orchestra.run.smithi148.stdout:checking for nftw... yes 2024-03-04T12:14:06.196 INFO:teuthology.orchestra.run.smithi148.stdout:checking for RLIMIT_NOFILE... yes 2024-03-04T12:14:06.227 INFO:teuthology.orchestra.run.smithi148.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-03-04T12:14:06.261 INFO:teuthology.orchestra.run.smithi148.stdout:checking for FICLONE... yes 2024-03-04T12:14:06.295 INFO:teuthology.orchestra.run.smithi148.stdout:checking for renameat2... yes 2024-03-04T12:14:06.328 INFO:teuthology.orchestra.run.smithi148.stdout:checking for reallocarray... yes 2024-03-04T12:14:06.358 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct mount_attr... yes 2024-03-04T12:14:06.391 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_qgroup_limit... yes 2024-03-04T12:14:06.423 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-03-04T12:14:06.455 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-03-04T12:14:06.487 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-03-04T12:14:06.519 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-03-04T12:14:06.551 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-03-04T12:14:06.583 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-03-04T12:14:06.609 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/btrfs.h usability... yes 2024-03-04T12:14:06.617 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/btrfs.h presence... yes 2024-03-04T12:14:06.617 INFO:teuthology.orchestra.run.smithi148.stdout:checking for linux/btrfs.h... yes 2024-03-04T12:14:06.655 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/btrfs_tree.h usability... yes 2024-03-04T12:14:06.664 INFO:teuthology.orchestra.run.smithi148.stdout:checking linux/btrfs_tree.h presence... yes 2024-03-04T12:14:06.664 INFO:teuthology.orchestra.run.smithi148.stdout:checking for linux/btrfs_tree.h... yes 2024-03-04T12:14:06.685 INFO:teuthology.orchestra.run.smithi148.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-03-04T12:14:06.706 INFO:teuthology.orchestra.run.smithi148.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-03-04T12:14:06.722 INFO:teuthology.orchestra.run.smithi148.stdout:configure: creating ./config.status 2024-03-04T12:14:07.015 INFO:teuthology.orchestra.run.smithi148.stdout:config.status: creating include/builddefs 2024-03-04T12:14:07.025 INFO:teuthology.orchestra.run.smithi148.stdout:config.status: creating include/config.h 2024-03-04T12:14:07.033 INFO:teuthology.orchestra.run.smithi148.stdout:config.status: executing libtool commands 2024-03-04T12:14:07.070 INFO:teuthology.orchestra.run.smithi148.stdout:Building include 2024-03-04T12:14:07.073 INFO:teuthology.orchestra.run.smithi148.stdout:Building lib 2024-03-04T12:14:07.076 INFO:teuthology.orchestra.run.smithi148.stdout: [LTDEP] 2024-03-04T12:14:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dataascii.lo 2024-03-04T12:14:07.315 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] databin.lo 2024-03-04T12:14:07.447 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] datapid.lo 2024-03-04T12:14:07.544 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] file_lock.lo 2024-03-04T12:14:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] forker.lo 2024-03-04T12:14:07.794 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] pattern.lo 2024-03-04T12:14:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] open_flags.lo 2024-03-04T12:14:08.032 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] random_range.lo 2024-03-04T12:14:08.237 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] string_to_tokens.lo 2024-03-04T12:14:08.329 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] str_to_bytes.lo 2024-03-04T12:14:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] tlibio.lo 2024-03-04T12:14:08.665 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] write_log.lo 2024-03-04T12:14:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] random.lo 2024-03-04T12:14:08.977 INFO:teuthology.orchestra.run.smithi148.stdout: [LD] 2024-03-04T12:14:09.058 INFO:teuthology.orchestra.run.smithi148.stdout:Building ltp 2024-03-04T12:14:09.060 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:09.192 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] doio 2024-03-04T12:14:09.802 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fsstress 2024-03-04T12:14:11.111 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fsx 2024-03-04T12:14:11.779 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] growfiles 2024-03-04T12:14:12.233 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] iogen 2024-03-04T12:14:12.573 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-stress 2024-03-04T12:14:12.954 INFO:teuthology.orchestra.run.smithi148.stdout:Building src 2024-03-04T12:14:12.960 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:14.674 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: [DEP] 2024-03-04T12:14:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dirstress 2024-03-04T12:14:15.152 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fill 2024-03-04T12:14:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fill2 2024-03-04T12:14:15.481 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] getpagesize 2024-03-04T12:14:15.596 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] holes 2024-03-04T12:14:15.764 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] lstat64 2024-03-04T12:14:15.921 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] nametest 2024-03-04T12:14:16.129 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] permname 2024-03-04T12:14:16.302 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] randholes 2024-03-04T12:14:16.520 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] runas 2024-03-04T12:14:16.668 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] truncfile 2024-03-04T12:14:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] usemem 2024-03-04T12:14:16.951 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] mmapcat 2024-03-04T12:14:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] append_reader 2024-03-04T12:14:17.203 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] append_writer 2024-03-04T12:14:17.331 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dirperf 2024-03-04T12:14:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] metaperf 2024-03-04T12:14:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] devzero 2024-03-04T12:14:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] feature 2024-03-04T12:14:18.104 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] alloc 2024-03-04T12:14:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fault 2024-03-04T12:14:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fstest 2024-03-04T12:14:18.626 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_access_root 2024-03-04T12:14:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] godown 2024-03-04T12:14:18.903 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] resvtest 2024-03-04T12:14:19.059 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] writemod 2024-03-04T12:14:19.182 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] writev_on_pagefault 2024-03-04T12:14:19.318 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] makeextents 2024-03-04T12:14:19.469 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] itrash 2024-03-04T12:14:19.598 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] rename 2024-03-04T12:14:19.719 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] multi_open_unlink 2024-03-04T12:14:19.860 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] unwritten_sync 2024-03-04T12:14:20.012 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] genhashnames 2024-03-04T12:14:20.177 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_holes 2024-03-04T12:14:20.301 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_writev 2024-03-04T12:14:20.427 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_truncate_cmtime 2024-03-04T12:14:20.561 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dirhash_collide 2024-03-04T12:14:20.717 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_rename_overwrite 2024-03-04T12:14:20.836 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] holetest 2024-03-04T12:14:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_truncate_self 2024-03-04T12:14:21.153 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] af_unix 2024-03-04T12:14:21.282 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_stale_pmd 2024-03-04T12:14:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_cow_race 2024-03-04T12:14:21.552 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_fallocate 2024-03-04T12:14:21.680 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fsync-err 2024-03-04T12:14:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_write_ro 2024-03-04T12:14:21.967 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_ext4_dax_journal_corruption 2024-03-04T12:14:22.102 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_ext4_dax_inline_corruption 2024-03-04T12:14:22.234 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_ofd_locks 2024-03-04T12:14:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_collision 2024-03-04T12:14:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] mmap-write-concurrent 2024-03-04T12:14:22.718 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_get_file_time 2024-03-04T12:14:22.843 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_create_short_dirs 2024-03-04T12:14:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_create_long_dirs 2024-03-04T12:14:23.128 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_enospc 2024-03-04T12:14:23.288 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_writev_overlap 2024-03-04T12:14:23.432 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] checkpoint_journal 2024-03-04T12:14:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] mmap-rw-fault 2024-03-04T12:14:23.722 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] allocstale 2024-03-04T12:14:23.868 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_cow_memory_failure 2024-03-04T12:14:24.015 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fake-dump-rootino 2024-03-04T12:14:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dio-buf-fault 2024-03-04T12:14:24.272 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] rewinddir-test 2024-03-04T12:14:24.417 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] readdir-while-renames 2024-03-04T12:14:24.556 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] xfsctl 2024-03-04T12:14:24.718 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] bstat 2024-03-04T12:14:24.898 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mtab 2024-03-04T12:14:24.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:14:24.978 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:14:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] getdevicesize 2024-03-04T12:14:25.180 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] preallo_rw_pattern_reader 2024-03-04T12:14:25.314 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] preallo_rw_pattern_writer 2024-03-04T12:14:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] ftrunc 2024-03-04T12:14:25.576 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] trunc 2024-03-04T12:14:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fs_perms 2024-03-04T12:14:25.873 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] testx 2024-03-04T12:14:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] looptest 2024-03-04T12:14:26.151 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] locktest 2024-03-04T12:14:26.432 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] unwritten_mmap 2024-03-04T12:14:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] bulkstat_unlink_test 2024-03-04T12:14:26.741 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] deduperace 2024-03-04T12:14:26.908 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] bulkstat_unlink_test_modified 2024-03-04T12:14:27.076 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_dir_offset 2024-03-04T12:14:27.201 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_futimens 2024-03-04T12:14:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_immutable 2024-03-04T12:14:27.879 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] stale_handle 2024-03-04T12:14:28.023 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] pwrite_mmap_blocked 2024-03-04T12:14:28.152 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_dir_offset2 2024-03-04T12:14:28.316 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] seek_sanity_test 2024-03-04T12:14:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] seek_copy_test 2024-03-04T12:14:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_readdir_1 2024-03-04T12:14:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_readdir_2 2024-03-04T12:14:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fsync-tester 2024-03-04T12:14:29.335 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] nsexec 2024-03-04T12:14:29.487 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] cloner 2024-03-04T12:14:29.638 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] renameat2 2024-03-04T12:14:29.781 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_getcwd 2024-03-04T12:14:29.916 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] e4compact 2024-03-04T12:14:30.090 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] test-nextquota 2024-03-04T12:14:30.236 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] punch-alternating 2024-03-04T12:14:30.381 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] attr-list-by-handle-cursor-test 2024-03-04T12:14:30.523 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] listxattr 2024-03-04T12:14:30.647 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dio-interleaved 2024-03-04T12:14:30.786 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_dir_type 2024-03-04T12:14:30.918 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dio-invalidate-cache 2024-03-04T12:14:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] stat_test 2024-03-04T12:14:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_encrypted_d_revalidate 2024-03-04T12:14:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] attr_replace_test 2024-03-04T12:14:31.631 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] swapon 2024-03-04T12:14:31.776 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] mkswap 2024-03-04T12:14:31.904 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_attr_corruption 2024-03-04T12:14:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_open_tmpfiles 2024-03-04T12:14:32.182 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fscrypt-crypt-util 2024-03-04T12:14:32.781 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] bulkstat_null_ocount 2024-03-04T12:14:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] splice-test 2024-03-04T12:14:33.070 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] chprojid_fail 2024-03-04T12:14:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] detached_mounts_propagation 2024-03-04T12:14:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] ext4_resize 2024-03-04T12:14:33.492 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_readdir_3 2024-03-04T12:14:33.659 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] splice2pipe 2024-03-04T12:14:33.799 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] uuid_ioctl 2024-03-04T12:14:33.930 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_snapshot_deleted_subvolume 2024-03-04T12:14:34.070 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fiemap-tester 2024-03-04T12:14:34.302 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_stripealign 2024-03-04T12:14:34.432 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] open_by_handle 2024-03-04T12:14:34.651 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] dbtest 2024-03-04T12:14:34.872 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_mmap_dio 2024-03-04T12:14:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] xfsfind 2024-03-04T12:14:35.175 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] t_reflink_read_race 2024-03-04T12:14:35.341 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] fssum 2024-03-04T12:14:35.737 INFO:teuthology.orchestra.run.smithi148.stdout:Building log-writes 2024-03-04T12:14:35.742 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] replay-log 2024-03-04T12:14:36.023 INFO:teuthology.orchestra.run.smithi148.stdout:Building perf 2024-03-04T12:14:36.026 INFO:teuthology.orchestra.run.smithi148.stdout:Building aio-dio-regress 2024-03-04T12:14:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aiocp 2024-03-04T12:14:36.234 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-append-write-fallocate-race 2024-03-04T12:14:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-append-write-read-race 2024-03-04T12:14:36.529 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-cow-race 2024-03-04T12:14:36.661 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-cycle-write 2024-03-04T12:14:36.821 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-eof-race 2024-03-04T12:14:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-extend-stat 2024-03-04T12:14:37.118 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-fcntl-race 2024-03-04T12:14:37.257 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-hole-filling-race 2024-03-04T12:14:37.386 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-invalidate-failure 2024-03-04T12:14:37.520 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-invalidate-readahead 2024-03-04T12:14:37.660 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aiodio_sparse2 2024-03-04T12:14:37.851 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-subblock-eof-read 2024-03-04T12:14:37.975 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-dio-write-verify 2024-03-04T12:14:38.141 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-03-04T12:14:38.254 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-03-04T12:14:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] aio-last-ref-held-by-io 2024-03-04T12:14:38.513 INFO:teuthology.orchestra.run.smithi148.stdout:Building vfs 2024-03-04T12:14:38.518 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] vfstest 2024-03-04T12:14:41.839 INFO:teuthology.orchestra.run.smithi148.stdout: [CC] mount-idmapped 2024-03-04T12:14:42.178 INFO:teuthology.orchestra.run.smithi148.stdout:Building m4 2024-03-04T12:14:42.182 INFO:teuthology.orchestra.run.smithi148.stdout:Building common 2024-03-04T12:14:42.185 INFO:teuthology.orchestra.run.smithi148.stdout:Building tests 2024-03-04T12:14:42.190 INFO:teuthology.orchestra.run.smithi148.stdout:Building btrfs 2024-03-04T12:14:42.196 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/btrfs/group.list 2024-03-04T12:14:42.255 INFO:teuthology.orchestra.run.smithi148.stdout:Building ceph 2024-03-04T12:14:42.259 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ceph/group.list 2024-03-04T12:14:42.274 INFO:teuthology.orchestra.run.smithi148.stdout:Building cifs 2024-03-04T12:14:42.278 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/cifs/group.list 2024-03-04T12:14:42.290 INFO:teuthology.orchestra.run.smithi148.stdout:Building ext4 2024-03-04T12:14:42.295 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ext4/group.list 2024-03-04T12:14:42.332 INFO:teuthology.orchestra.run.smithi148.stdout:Building f2fs 2024-03-04T12:14:42.337 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/f2fs/group.list 2024-03-04T12:14:42.353 INFO:teuthology.orchestra.run.smithi148.stdout:Building generic 2024-03-04T12:14:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/generic/group.list 2024-03-04T12:14:42.453 INFO:teuthology.orchestra.run.smithi148.stdout:Building nfs 2024-03-04T12:14:42.458 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/nfs/group.list 2024-03-04T12:14:42.472 INFO:teuthology.orchestra.run.smithi148.stdout:Building ocfs2 2024-03-04T12:14:42.477 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ocfs2/group.list 2024-03-04T12:14:42.490 INFO:teuthology.orchestra.run.smithi148.stdout:Building overlay 2024-03-04T12:14:42.495 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/overlay/group.list 2024-03-04T12:14:42.534 INFO:teuthology.orchestra.run.smithi148.stdout:Building perf 2024-03-04T12:14:42.539 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/perf/group.list 2024-03-04T12:14:42.550 INFO:teuthology.orchestra.run.smithi148.stdout:Building selftest 2024-03-04T12:14:42.555 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/selftest/group.list 2024-03-04T12:14:42.566 INFO:teuthology.orchestra.run.smithi148.stdout:Building shared 2024-03-04T12:14:42.571 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/shared/group.list 2024-03-04T12:14:42.588 INFO:teuthology.orchestra.run.smithi148.stdout:Building tmpfs 2024-03-04T12:14:42.593 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/tmpfs/group.list 2024-03-04T12:14:42.605 INFO:teuthology.orchestra.run.smithi148.stdout:Building udf 2024-03-04T12:14:42.610 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/udf/group.list 2024-03-04T12:14:42.621 INFO:teuthology.orchestra.run.smithi148.stdout:Building xfs 2024-03-04T12:14:42.627 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/xfs/group.list 2024-03-04T12:14:42.724 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo make install) 2024-03-04T12:14:42.803 INFO:teuthology.orchestra.run.smithi148.stdout:Building include 2024-03-04T12:14:42.806 INFO:teuthology.orchestra.run.smithi148.stdout:Building lib 2024-03-04T12:14:42.810 INFO:teuthology.orchestra.run.smithi148.stdout:Building ltp 2024-03-04T12:14:42.813 INFO:teuthology.orchestra.run.smithi148.stdout:Building src 2024-03-04T12:14:42.847 INFO:teuthology.orchestra.run.smithi148.stdout:Building log-writes 2024-03-04T12:14:42.850 INFO:teuthology.orchestra.run.smithi148.stdout:Building perf 2024-03-04T12:14:42.853 INFO:teuthology.orchestra.run.smithi148.stdout:Building aio-dio-regress 2024-03-04T12:14:42.857 INFO:teuthology.orchestra.run.smithi148.stdout:Building vfs 2024-03-04T12:14:42.860 INFO:teuthology.orchestra.run.smithi148.stdout:Building m4 2024-03-04T12:14:42.864 INFO:teuthology.orchestra.run.smithi148.stdout:Building common 2024-03-04T12:14:42.867 INFO:teuthology.orchestra.run.smithi148.stdout:Building tests 2024-03-04T12:14:42.872 INFO:teuthology.orchestra.run.smithi148.stdout:Building btrfs 2024-03-04T12:14:42.878 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/btrfs/group.list 2024-03-04T12:14:42.936 INFO:teuthology.orchestra.run.smithi148.stdout:Building ceph 2024-03-04T12:14:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ceph/group.list 2024-03-04T12:14:42.955 INFO:teuthology.orchestra.run.smithi148.stdout:Building cifs 2024-03-04T12:14:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/cifs/group.list 2024-03-04T12:14:42.971 INFO:teuthology.orchestra.run.smithi148.stdout:Building ext4 2024-03-04T12:14:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ext4/group.list 2024-03-04T12:14:43.013 INFO:teuthology.orchestra.run.smithi148.stdout:Building f2fs 2024-03-04T12:14:43.018 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/f2fs/group.list 2024-03-04T12:14:43.035 INFO:teuthology.orchestra.run.smithi148.stdout:Building generic 2024-03-04T12:14:43.041 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/generic/group.list 2024-03-04T12:14:43.134 INFO:teuthology.orchestra.run.smithi148.stdout:Building nfs 2024-03-04T12:14:43.138 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/nfs/group.list 2024-03-04T12:14:43.152 INFO:teuthology.orchestra.run.smithi148.stdout:Building ocfs2 2024-03-04T12:14:43.157 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ocfs2/group.list 2024-03-04T12:14:43.170 INFO:teuthology.orchestra.run.smithi148.stdout:Building overlay 2024-03-04T12:14:43.175 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/overlay/group.list 2024-03-04T12:14:43.214 INFO:teuthology.orchestra.run.smithi148.stdout:Building perf 2024-03-04T12:14:43.220 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/perf/group.list 2024-03-04T12:14:43.230 INFO:teuthology.orchestra.run.smithi148.stdout:Building selftest 2024-03-04T12:14:43.235 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/selftest/group.list 2024-03-04T12:14:43.247 INFO:teuthology.orchestra.run.smithi148.stdout:Building shared 2024-03-04T12:14:43.252 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/shared/group.list 2024-03-04T12:14:43.269 INFO:teuthology.orchestra.run.smithi148.stdout:Building tmpfs 2024-03-04T12:14:43.273 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/tmpfs/group.list 2024-03-04T12:14:43.286 INFO:teuthology.orchestra.run.smithi148.stdout:Building udf 2024-03-04T12:14:43.291 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/udf/group.list 2024-03-04T12:14:43.303 INFO:teuthology.orchestra.run.smithi148.stdout:Building xfs 2024-03-04T12:14:43.309 INFO:teuthology.orchestra.run.smithi148.stdout: [GROUP] /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/xfs/group.list 2024-03-04T12:14:43.401 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-03-04T12:14:43.403 INFO:teuthology.orchestra.run.smithi148.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-04T12:14:43.403 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-03-04T12:14:43.405 INFO:teuthology.orchestra.run.smithi148.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-04T12:14:43.405 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-03-04T12:14:43.407 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-03-04T12:14:43.415 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-03-04T12:14:43.482 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-03-04T12:14:43.490 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-03-04T12:14:43.507 INFO:teuthology.orchestra.run.smithi148.stdout:Building log-writes 2024-03-04T12:14:43.511 INFO:teuthology.orchestra.run.smithi148.stdout:Building perf 2024-03-04T12:14:43.513 INFO:teuthology.orchestra.run.smithi148.stdout:Building aio-dio-regress 2024-03-04T12:14:43.517 INFO:teuthology.orchestra.run.smithi148.stdout:Building vfs 2024-03-04T12:14:43.520 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake -C log-writes install 2024-03-04T12:14:43.522 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-03-04T12:14:43.530 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-03-04T12:14:43.538 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake -C perf install 2024-03-04T12:14:43.540 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-03-04T12:14:43.548 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-03-04T12:14:43.568 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-03-04T12:14:43.570 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-03-04T12:14:43.578 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-03-04T12:14:43.626 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake -C vfs install 2024-03-04T12:14:43.627 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-03-04T12:14:43.636 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-03-04T12:14:43.647 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-03-04T12:14:43.655 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-03-04T12:14:44.766 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-03-04T12:14:44.857 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-03-04T12:14:44.881 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-03-04T12:14:44.883 INFO:teuthology.orchestra.run.smithi148.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-04T12:14:44.883 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-03-04T12:14:44.884 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-03-04T12:14:44.892 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-03-04T12:14:45.018 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-03-04T12:14:45.020 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/btrfs/ install 2024-03-04T12:14:45.023 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-03-04T12:14:45.031 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 304 305 306 307 308 309 310 /var/lib/xfstests/tests/btrfs 2024-03-04T12:14:45.780 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-03-04T12:14:45.788 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out /var/lib/xfstests/tests/btrfs 2024-03-04T12:14:46.540 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ceph/ install 2024-03-04T12:14:46.541 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-03-04T12:14:46.549 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-03-04T12:14:46.567 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-03-04T12:14:46.575 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-03-04T12:14:46.593 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/cifs/ install 2024-03-04T12:14:46.594 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-03-04T12:14:46.603 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-03-04T12:14:46.611 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-03-04T12:14:46.619 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-03-04T12:14:46.627 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ext4/ install 2024-03-04T12:14:46.629 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-03-04T12:14:46.637 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-03-04T12:14:46.806 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-03-04T12:14:46.814 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-03-04T12:14:46.992 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/f2fs/ install 2024-03-04T12:14:46.997 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-03-04T12:14:47.002 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-03-04T12:14:47.012 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-03-04T12:14:47.020 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-03-04T12:14:47.031 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/generic/ install 2024-03-04T12:14:47.036 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-03-04T12:14:47.044 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-03-04T12:14:48.846 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-03-04T12:14:48.855 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-03-04T12:14:50.671 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/nfs/ install 2024-03-04T12:14:50.673 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-03-04T12:14:50.682 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-03-04T12:14:50.690 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-03-04T12:14:50.698 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-03-04T12:14:50.706 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/ocfs2/ install 2024-03-04T12:14:50.707 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-03-04T12:14:50.715 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-03-04T12:14:50.724 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-03-04T12:14:50.732 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-03-04T12:14:50.741 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/overlay/ install 2024-03-04T12:14:50.743 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-03-04T12:14:50.751 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-03-04T12:14:51.008 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-03-04T12:14:51.016 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-03-04T12:14:51.274 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/perf/ install 2024-03-04T12:14:51.275 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-03-04T12:14:51.284 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-03-04T12:14:51.292 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-03-04T12:14:51.300 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-03-04T12:14:51.309 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/selftest/ install 2024-03-04T12:14:51.310 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-03-04T12:14:51.318 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-03-04T12:14:51.339 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-03-04T12:14:51.347 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-03-04T12:14:51.367 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/shared/ install 2024-03-04T12:14:51.369 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-03-04T12:14:51.377 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-03-04T12:14:51.390 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-03-04T12:14:51.398 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-03-04T12:14:51.412 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/tmpfs/ install 2024-03-04T12:14:51.413 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-03-04T12:14:51.421 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-03-04T12:14:51.429 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-03-04T12:14:51.438 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-03-04T12:14:51.446 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/udf/ install 2024-03-04T12:14:51.448 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-03-04T12:14:51.456 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-03-04T12:14:51.464 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-03-04T12:14:51.472 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-03-04T12:14:51.481 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ATjWjXU3R3xfstests-dev/tests/xfs/ install 2024-03-04T12:14:51.486 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-03-04T12:14:51.494 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-03-04T12:14:53.190 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-03-04T12:14:53.198 INFO:teuthology.orchestra.run.smithi148.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-03-04T12:14:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-03-04T12:14:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-03-04T12:14:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-03-04T12:14:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-03-04T12:14:54.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T12:14:54.955 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec ls tests/generic) 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:001 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:001.out 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:002 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:002.out 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:003 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:003.out 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:004 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:004.out 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:005 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:005.out 2024-03-04T12:14:55.012 INFO:teuthology.orchestra.run.smithi148.stdout:006 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:006.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:007 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:007.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:008 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:008.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:009 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:009.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:010 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:010.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:011 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:011.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:012 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:012.out 2024-03-04T12:14:55.013 INFO:teuthology.orchestra.run.smithi148.stdout:013 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:013.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:014 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:014.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:015 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:015.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:016 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:016.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:017 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:017.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:018 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:018.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:019 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:019.out 2024-03-04T12:14:55.014 INFO:teuthology.orchestra.run.smithi148.stdout:020 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:020.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:021 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:021.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:022 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:022.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:023 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:023.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:024 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:024.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:025 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:025.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:026 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:026.out 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:027 2024-03-04T12:14:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:027.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:028 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:028.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:029 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:029.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:030 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:030.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:031 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:031.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:032 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:032.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:033 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:033.out 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:034 2024-03-04T12:14:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:034.out 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:035 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:035.cfg 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:035.out.default 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:035.out.nfs 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:036 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:036.out 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:037 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:037.out 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:038 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:038.out 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:039 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:039.out 2024-03-04T12:14:55.017 INFO:teuthology.orchestra.run.smithi148.stdout:040 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:040.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:041 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:041.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:042 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:042.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:043 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:043.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:044 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:044.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:045 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:045.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:046 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:046.out 2024-03-04T12:14:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:047 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:047.out 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:048 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:048.out 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:049 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:049.out 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:050 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:050.cfg 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:050.out.default 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:050.out.nojournal 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:050.out.xfsquota 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:051 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:051.out 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:052 2024-03-04T12:14:55.019 INFO:teuthology.orchestra.run.smithi148.stdout:052.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:053 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:053.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:054 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:054.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:055 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:055.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:056 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:056.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:057 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:057.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:058 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:058.out 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:059 2024-03-04T12:14:55.020 INFO:teuthology.orchestra.run.smithi148.stdout:059.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:060 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:060.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:061 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:061.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:062 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:062.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:063 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:063.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:064 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:064.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:065 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:065.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:066 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:066.out 2024-03-04T12:14:55.021 INFO:teuthology.orchestra.run.smithi148.stdout:067 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:067.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:068 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:068.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:069 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:069.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:070 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:070.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:071 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:071.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:072 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:072.out 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:073 2024-03-04T12:14:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:073.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:074 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:074.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:075 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:075.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:076 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:076.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:077 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:077.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:078 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:078.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:079 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:079.out 2024-03-04T12:14:55.023 INFO:teuthology.orchestra.run.smithi148.stdout:080 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:080.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:081 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:081.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:082 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:082.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:083 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:083.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:084 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:084.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:085 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:085.out 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:086 2024-03-04T12:14:55.024 INFO:teuthology.orchestra.run.smithi148.stdout:086.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:087 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:087.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:088 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:088.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:089 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:089.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:090 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:090.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:091 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:091.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:092 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:092.out 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:093 2024-03-04T12:14:55.025 INFO:teuthology.orchestra.run.smithi148.stdout:093.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:094 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:094.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:095 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:095.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:096 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:096.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:097 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:097.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:098 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:098.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:099 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:099.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:100 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:100.out 2024-03-04T12:14:55.026 INFO:teuthology.orchestra.run.smithi148.stdout:101 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:101.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:102 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:102.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:103 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:103.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:104 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:104.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:105 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:105.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:106 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:106.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:107 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:107.out 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:108 2024-03-04T12:14:55.027 INFO:teuthology.orchestra.run.smithi148.stdout:108.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:109 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:109.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:110 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:110.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:111 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:111.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:112 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:112.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:113 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:113.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:114 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:114.out 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:115 2024-03-04T12:14:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:115.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:116 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:116.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:117 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:117.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:118 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:118.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:119 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:119.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:120 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:120.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:121 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:121.out 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:122 2024-03-04T12:14:55.029 INFO:teuthology.orchestra.run.smithi148.stdout:122.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:123 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:123.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:124 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:124.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:125 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:125.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:126 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:126.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:127 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:127.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:128 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:128.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:129 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:129.out 2024-03-04T12:14:55.030 INFO:teuthology.orchestra.run.smithi148.stdout:130 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:130.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:131 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:131.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:132 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:132.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:133 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:133.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:134 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:134.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:135 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:135.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:136 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:136.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:137 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:137.out 2024-03-04T12:14:55.031 INFO:teuthology.orchestra.run.smithi148.stdout:138 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:138.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:139 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:139.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:140 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:140.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:141 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:141.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:142 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:142.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:143 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:143.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:144 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:144.out 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:145 2024-03-04T12:14:55.032 INFO:teuthology.orchestra.run.smithi148.stdout:145.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:146 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:146.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:147 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:147.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:148 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:148.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:149 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:149.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:150 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:150.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:151 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:151.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:152 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:152.out 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:153 2024-03-04T12:14:55.033 INFO:teuthology.orchestra.run.smithi148.stdout:153.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:154 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:154.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:155 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:155.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:156 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:156.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:157 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:157.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:158 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:158.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:159 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:159.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:160 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:160.out 2024-03-04T12:14:55.034 INFO:teuthology.orchestra.run.smithi148.stdout:161 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:161.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:162 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:162.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:163 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:163.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:164 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:164.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:165 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:165.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:166 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:166.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:167 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:167.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:168 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:168.out 2024-03-04T12:14:55.035 INFO:teuthology.orchestra.run.smithi148.stdout:169 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:169.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:170 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:170.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:171 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:171.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:172 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:172.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:173 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:173.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:174 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:174.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:175 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:175.out 2024-03-04T12:14:55.036 INFO:teuthology.orchestra.run.smithi148.stdout:176 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:176.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:177 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:177.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:178 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:178.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:179 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:179.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:180 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:180.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:181 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:181.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:182 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:182.out 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:183 2024-03-04T12:14:55.037 INFO:teuthology.orchestra.run.smithi148.stdout:183.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:184 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:184.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:185 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:185.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:186 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:186.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:187 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:187.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:188 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:188.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:189 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:189.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:190 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:190.out 2024-03-04T12:14:55.038 INFO:teuthology.orchestra.run.smithi148.stdout:191 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:191.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:192 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:192.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:193 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:193.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:194 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:194.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:195 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:195.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:196 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:196.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:197 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:197.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:198 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:198.out 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:199 2024-03-04T12:14:55.039 INFO:teuthology.orchestra.run.smithi148.stdout:199.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:200 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:200.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:201 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:201.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:202 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:202.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:203 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:203.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:204 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:204.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:205 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:205.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:206 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:206.out 2024-03-04T12:14:55.040 INFO:teuthology.orchestra.run.smithi148.stdout:207 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:207.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:208 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:208.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:209 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:209.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:210 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:210.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:211 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:211.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:212 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:212.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:213 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:213.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:214 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:214.out 2024-03-04T12:14:55.041 INFO:teuthology.orchestra.run.smithi148.stdout:215 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:215.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:216 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:216.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:217 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:217.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:218 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:218.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:219 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:219.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:220 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:220.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:221 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:221.out 2024-03-04T12:14:55.042 INFO:teuthology.orchestra.run.smithi148.stdout:222 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:222.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:223 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:223.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:224 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:224.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:225 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:225.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:226 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:226.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:227 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:227.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:228 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:228.out 2024-03-04T12:14:55.043 INFO:teuthology.orchestra.run.smithi148.stdout:229 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:229.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:230 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:230.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:231 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:231.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:232 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:232.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:233 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:233.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:234 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:234.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:235 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:235.out 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:236 2024-03-04T12:14:55.044 INFO:teuthology.orchestra.run.smithi148.stdout:236.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:237 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:237.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:238 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:238.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:239 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:239.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:240 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:240.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:241 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:241.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:242 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:242.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:243 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:243.out 2024-03-04T12:14:55.045 INFO:teuthology.orchestra.run.smithi148.stdout:244 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:244.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:245 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:245.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:246 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:246.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:247 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:247.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:248 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:248.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:249 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:249.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:250 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:250.out 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:251 2024-03-04T12:14:55.046 INFO:teuthology.orchestra.run.smithi148.stdout:251.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:252 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:252.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:253 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:253.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:254 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:254.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:255 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:255.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:256 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:256.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:257 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:257.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:258 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:258.out 2024-03-04T12:14:55.047 INFO:teuthology.orchestra.run.smithi148.stdout:259 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:259.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:260 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:260.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:261 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:261.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:262 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:262.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:263 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:263.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:264 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:264.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:265 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:265.out 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:266 2024-03-04T12:14:55.048 INFO:teuthology.orchestra.run.smithi148.stdout:266.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:267 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:267.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:268 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:268.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:269 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:269.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:270 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:270.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:271 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:271.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:272 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:272.out 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:273 2024-03-04T12:14:55.049 INFO:teuthology.orchestra.run.smithi148.stdout:273.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:274 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:274.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:275 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:275.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:276 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:276.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:277 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:277.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:278 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:278.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:279 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:279.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:280 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:280.out 2024-03-04T12:14:55.050 INFO:teuthology.orchestra.run.smithi148.stdout:281 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:281.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:282 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:282.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:283 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:283.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:284 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:284.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:285 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:285.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:286 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:286.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:287 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:287.out 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:288 2024-03-04T12:14:55.051 INFO:teuthology.orchestra.run.smithi148.stdout:288.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:289 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:289.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:290 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:290.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:291 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:291.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:292 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:292.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:293 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:293.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:294 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:294.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:295 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:295.out 2024-03-04T12:14:55.052 INFO:teuthology.orchestra.run.smithi148.stdout:296 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:296.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:297 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:297.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:298 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:298.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:299 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:299.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:300 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:300.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:301 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:301.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:302 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:302.out 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:303 2024-03-04T12:14:55.053 INFO:teuthology.orchestra.run.smithi148.stdout:303.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:304 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:304.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:305 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:305.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:306 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:306.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:307 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:307.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:308 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:308.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:309 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:309.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:310 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:310.out 2024-03-04T12:14:55.054 INFO:teuthology.orchestra.run.smithi148.stdout:311 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:311.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:312 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:312.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:313 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:313.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:314 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:314.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:315 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:315.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:316 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:316.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:317 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:317.out 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:318 2024-03-04T12:14:55.055 INFO:teuthology.orchestra.run.smithi148.stdout:318.out 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:319 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:319.out 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:320 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:320.out 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:321 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:321.out 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:322 2024-03-04T12:14:55.056 INFO:teuthology.orchestra.run.smithi148.stdout:322.out 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:323 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:323.out 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:324 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:324.out 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:325 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:325.out 2024-03-04T12:14:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:326 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:326.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:327 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:327.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:328 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:328.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:329 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:329.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:330 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:330.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:331 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:331.out 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:332 2024-03-04T12:14:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:332.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:333 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:333.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:334 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:334.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:335 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:335.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:336 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:336.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:337 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:337.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:338 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:338.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:339 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:339.out 2024-03-04T12:14:55.059 INFO:teuthology.orchestra.run.smithi148.stdout:340 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:340.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:341 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:341.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:342 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:342.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:343 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:343.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:344 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:344.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:345 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:345.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:346 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:346.out 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:347 2024-03-04T12:14:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:347.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:348 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:348.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:349 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:349.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:350 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:350.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:351 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:351.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:352 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:352.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:353 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:353.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:354 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:354.out 2024-03-04T12:14:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:355 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:355.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:356 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:356.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:357 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:357.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:358 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:358.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:359 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:359.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:360 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:360.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:361 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:361.out 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:362 2024-03-04T12:14:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:362.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:363 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:363.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:364 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:364.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:365 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:365.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:366 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:366.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:367 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:367.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:368 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:368.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:369 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:369.out 2024-03-04T12:14:55.063 INFO:teuthology.orchestra.run.smithi148.stdout:370 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:370.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:371 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:371.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:372 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:372.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:373 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:373.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:374 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:374.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:375 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:375.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:376 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:376.out 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:377 2024-03-04T12:14:55.064 INFO:teuthology.orchestra.run.smithi148.stdout:377.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:378 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:378.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:379 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:379.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:380 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:380.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:381 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:381.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:382 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:382.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:383 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:383.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:384 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:384.out 2024-03-04T12:14:55.065 INFO:teuthology.orchestra.run.smithi148.stdout:385 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:385.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:386 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:386.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:387 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:387.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:388 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:388.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:389 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:389.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:390 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:390.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:391 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:391.out 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:392 2024-03-04T12:14:55.066 INFO:teuthology.orchestra.run.smithi148.stdout:392.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:393 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:393.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:394 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:394.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:395 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:395.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:396 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:396.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:397 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:397.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:398 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:398.out 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:399 2024-03-04T12:14:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:399.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:400 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:400.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:401 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:401.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:402 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:402.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:403 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:403.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:404 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:404.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:405 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:405.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:406 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:406.out 2024-03-04T12:14:55.068 INFO:teuthology.orchestra.run.smithi148.stdout:407 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:407.out 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:408 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:408.out 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:409 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:409.out 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:410 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:410.out 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:411 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:411.out 2024-03-04T12:14:55.069 INFO:teuthology.orchestra.run.smithi148.stdout:412 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:412.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:413 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:413.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:414 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:414.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:415 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:415.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:416 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:416.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:417 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:417.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:418 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:418.out 2024-03-04T12:14:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:419 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:419.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:420 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:420.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:421 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:421.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:422 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:422.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:423 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:423.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:424 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:424.out 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:425 2024-03-04T12:14:55.071 INFO:teuthology.orchestra.run.smithi148.stdout:425.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:426 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:426.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:427 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:427.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:428 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:428.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:429 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:429.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:430 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:430.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:431 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:431.out 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:432 2024-03-04T12:14:55.072 INFO:teuthology.orchestra.run.smithi148.stdout:432.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:433 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:433.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:434 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:434.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:435 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:435.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:436 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:436.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:437 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:437.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:438 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:438.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:439 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:439.out 2024-03-04T12:14:55.073 INFO:teuthology.orchestra.run.smithi148.stdout:440 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:440.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:441 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:441.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:442 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:442.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:443 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:443.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:444 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:444.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:445 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:445.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:446 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:446.out 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:447 2024-03-04T12:14:55.074 INFO:teuthology.orchestra.run.smithi148.stdout:447.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:448 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:448.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:449 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:449.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:450 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:450.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:451 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:451.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:452 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:452.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:453 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:453.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:454 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:454.out 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:455 2024-03-04T12:14:55.075 INFO:teuthology.orchestra.run.smithi148.stdout:455.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:456 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:456.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:457 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:457.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:458 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:458.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:459 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:459.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:460 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:460.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:461 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:461.out 2024-03-04T12:14:55.076 INFO:teuthology.orchestra.run.smithi148.stdout:462 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:462.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:463 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:463.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:464 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:464.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:465 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:465.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:466 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:466.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:467 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:467.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:468 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:468.out 2024-03-04T12:14:55.077 INFO:teuthology.orchestra.run.smithi148.stdout:469 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:469.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:470 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:470.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:471 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:471.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:472 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:472.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:473 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:473.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:474 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:474.out 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:475 2024-03-04T12:14:55.078 INFO:teuthology.orchestra.run.smithi148.stdout:475.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:476 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:476.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:477 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:477.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:478 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:478.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:479 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:479.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:480 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:480.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:481 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:481.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:482 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:482.out 2024-03-04T12:14:55.079 INFO:teuthology.orchestra.run.smithi148.stdout:483 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:483.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:484 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:484.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:485 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:485.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:486 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:486.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:487 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:487.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:488 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:488.out 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:489 2024-03-04T12:14:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:489.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:490 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:490.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:491 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:491.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:492 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:492.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:493 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:493.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:494 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:494.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:495 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:495.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:496 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:496.out 2024-03-04T12:14:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:497 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:497.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:498 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:498.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:499 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:499.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:500 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:500.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:501 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:501.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:502 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:502.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:503 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:503.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:504 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:504.out 2024-03-04T12:14:55.082 INFO:teuthology.orchestra.run.smithi148.stdout:505 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:505.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:506 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:506.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:507 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:507.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:508 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:508.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:509 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:509.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:510 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:510.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:511 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:511.out 2024-03-04T12:14:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:512 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:512.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:513 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:513.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:514 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:514.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:515 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:515.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:516 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:516.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:517 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:517.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:518 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:518.out 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:519 2024-03-04T12:14:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:519.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:520 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:520.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:521 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:521.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:522 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:522.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:523 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:523.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:524 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:524.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:525 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:525.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:526 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:526.out 2024-03-04T12:14:55.085 INFO:teuthology.orchestra.run.smithi148.stdout:527 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:527.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:528 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:528.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:529 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:529.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:530 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:530.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:531 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:531.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:532 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:532.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:533 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:533.out 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:534 2024-03-04T12:14:55.086 INFO:teuthology.orchestra.run.smithi148.stdout:534.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:535 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:535.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:536 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:536.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:537 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:537.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:538 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:538.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:539 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:539.out 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:540 2024-03-04T12:14:55.087 INFO:teuthology.orchestra.run.smithi148.stdout:540.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:541 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:541.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:542 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:542.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:543 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:543.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:544 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:544.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:545 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:545.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:546 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:546.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:547 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:547.out 2024-03-04T12:14:55.088 INFO:teuthology.orchestra.run.smithi148.stdout:548 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:548.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:549 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:549.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:550 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:550.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:551 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:551.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:552 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:552.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:553 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:553.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:554 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:554.out 2024-03-04T12:14:55.089 INFO:teuthology.orchestra.run.smithi148.stdout:555 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:555.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:556 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:556.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:557 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:557.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:558 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:558.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:559 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:559.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:560 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:560.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:561 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:561.out 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:562 2024-03-04T12:14:55.090 INFO:teuthology.orchestra.run.smithi148.stdout:562.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:563 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:563.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:564 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:564.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:565 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:565.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:566 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:566.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:567 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:567.out 2024-03-04T12:14:55.091 INFO:teuthology.orchestra.run.smithi148.stdout:568 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:568.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:569 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:569.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:570 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:570.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:571 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:571.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:572 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:572.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:573 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:573.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:574 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:574.out 2024-03-04T12:14:55.092 INFO:teuthology.orchestra.run.smithi148.stdout:575 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:575.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:576 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:576.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:577 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:577.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:578 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:578.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:579 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:579.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:580 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:580.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:581 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:581.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:582 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:582.out 2024-03-04T12:14:55.093 INFO:teuthology.orchestra.run.smithi148.stdout:583 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:583.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:584 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:584.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:585 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:585.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:586 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:586.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:587 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:587.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:588 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:588.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:589 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:589.out 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:590 2024-03-04T12:14:55.094 INFO:teuthology.orchestra.run.smithi148.stdout:590.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:591 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:591.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:592 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:592.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:593 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:593.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:594 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:594.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:595 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:595.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:596 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:596.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:597 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:597.out 2024-03-04T12:14:55.095 INFO:teuthology.orchestra.run.smithi148.stdout:598 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:598.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:599 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:599.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:600 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:600.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:601 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:601.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:602 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:602.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:603 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:603.out 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:604 2024-03-04T12:14:55.096 INFO:teuthology.orchestra.run.smithi148.stdout:604.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:605 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:605.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:606 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:606.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:607 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:607.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:608 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:608.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:609 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:609.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:610 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:610.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:611 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:611.out 2024-03-04T12:14:55.097 INFO:teuthology.orchestra.run.smithi148.stdout:612 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:612.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:613 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:613.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:614 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:614.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:615 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:615.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:616 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:616.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:617 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:617.out 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:618 2024-03-04T12:14:55.098 INFO:teuthology.orchestra.run.smithi148.stdout:618.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:619 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:619.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:620 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:620.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:621 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:621.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:622 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:622.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:623 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:623.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:624 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:624.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:625 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:625.out 2024-03-04T12:14:55.099 INFO:teuthology.orchestra.run.smithi148.stdout:626 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:626.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:627 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:627.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:628 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:628.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:629 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:629.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:630 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:630.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:631 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:631.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:632 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:632.out 2024-03-04T12:14:55.100 INFO:teuthology.orchestra.run.smithi148.stdout:633 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:633.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:634 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:634.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:635 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:635.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:636 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:636.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:637 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:637.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:638 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:638.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:639 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:639.out 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:640 2024-03-04T12:14:55.101 INFO:teuthology.orchestra.run.smithi148.stdout:640.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:641 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:641.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:642 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:642.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:643 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:643.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:644 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:644.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:645 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:645.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:646 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:646.out 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:647 2024-03-04T12:14:55.102 INFO:teuthology.orchestra.run.smithi148.stdout:647.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:648 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:648.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:649 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:649.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:650 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:650.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:651 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:651.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:652 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:652.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:653 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:653.out 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:654 2024-03-04T12:14:55.103 INFO:teuthology.orchestra.run.smithi148.stdout:654.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:655 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:655.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:656 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:656.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:657 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:657.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:658 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:658.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:659 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:659.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:660 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:660.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:661 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:661.out 2024-03-04T12:14:55.104 INFO:teuthology.orchestra.run.smithi148.stdout:662 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:662.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:663 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:663.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:664 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:664.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:665 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:665.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:666 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:666.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:667 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:667.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:668 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:668.out 2024-03-04T12:14:55.105 INFO:teuthology.orchestra.run.smithi148.stdout:669 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:669.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:670 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:670.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:671 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:671.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:672 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:672.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:673 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:673.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:674 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:674.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:675 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:675.out 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:676 2024-03-04T12:14:55.106 INFO:teuthology.orchestra.run.smithi148.stdout:676.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:677 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:677.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:678 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:678.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:679 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:679.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:680 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:680.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:681 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:681.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:682 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:682.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:683 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:683.out 2024-03-04T12:14:55.107 INFO:teuthology.orchestra.run.smithi148.stdout:684 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:684.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:685 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:685.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:686 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:686.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:687 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:687.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:688 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:688.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:689 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:689.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:690 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:690.out 2024-03-04T12:14:55.108 INFO:teuthology.orchestra.run.smithi148.stdout:691 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:691.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:692 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:692.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:693 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:693.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:694 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:694.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:695 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:695.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:696 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:696.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:697 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:697.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:698 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:698.out 2024-03-04T12:14:55.109 INFO:teuthology.orchestra.run.smithi148.stdout:699 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:699.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:700 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:700.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:701 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:701.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:702 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:702.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:703 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:703.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:704 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:704.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:705 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:705.out 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:706 2024-03-04T12:14:55.110 INFO:teuthology.orchestra.run.smithi148.stdout:706.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:707 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:707.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:708 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:708.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:709 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:709.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:710 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:710.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:711 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:711.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:712 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:712.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:713 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:713.out 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:714 2024-03-04T12:14:55.111 INFO:teuthology.orchestra.run.smithi148.stdout:714.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:715 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:715.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:716 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:716.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:717 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:717.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:718 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:718.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:719 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:719.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:720 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:720.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:721 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:721.out 2024-03-04T12:14:55.112 INFO:teuthology.orchestra.run.smithi148.stdout:722 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:722.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:723 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:723.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:724 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:724.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:725 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:725.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:726 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:726.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:727 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:727.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:728 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:728.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:729 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:729.out 2024-03-04T12:14:55.113 INFO:teuthology.orchestra.run.smithi148.stdout:730 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:730.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:731 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:731.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:732 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:732.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:733 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:733.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:734 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:734.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:735 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:735.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:736 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:736.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:737 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:737.out 2024-03-04T12:14:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:738 2024-03-04T12:14:55.115 INFO:teuthology.orchestra.run.smithi148.stdout:738.out 2024-03-04T12:14:55.115 INFO:teuthology.orchestra.run.smithi148.stdout:739 2024-03-04T12:14:55.115 INFO:teuthology.orchestra.run.smithi148.stdout:739.out 2024-03-04T12:14:55.115 INFO:teuthology.orchestra.run.smithi148.stdout:group.list 2024-03-04T12:14:55.115 INFO:teuthology.orchestra.run.smithi148.stdout:Makefile 2024-03-04T12:14:55.115 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-03-04T12:14:55.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:14:55.118 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:14:55.292 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:14:55.296 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:14:55.296 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:14:55.297 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:14:55.297 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:14:58.972 INFO:teuthology.orchestra.run.smithi148.stdout:generic/001 3s 2024-03-04T12:14:58.977 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/001 2024-03-04T12:14:58.977 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:14:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:14:59.249 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:14:59.250 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-03-04T12:14:59.428 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:14:59.432 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:14:59.432 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:14:59.433 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:14:59.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:14:59.780 INFO:teuthology.orchestra.run.smithi148.stdout:generic/002 0s 2024-03-04T12:14:59.785 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/002 2024-03-04T12:14:59.785 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:14:59.787 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:14:59.855 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:14:59.856 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-03-04T12:15:00.134 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:15:00.138 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:15:00.138 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:15:00.139 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:15:00.139 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:00.340 INFO:teuthology.orchestra.run.smithi148.stdout:generic/003 [not run] atime not maintained by ceph 2024-03-04T12:15:00.344 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/003 2024-03-04T12:15:00.344 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/003 2024-03-04T12:15:00.344 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:15:00.346 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:00.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:15:00.357 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-03-04T12:15:00.532 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:15:00.536 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:15:00.536 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:15:00.537 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:15:00.537 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:00.738 INFO:teuthology.orchestra.run.smithi148.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-03-04T12:15:00.741 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/004 2024-03-04T12:15:00.742 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/004 2024-03-04T12:15:00.742 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:15:00.744 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:00.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:15:00.769 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-03-04T12:15:00.950 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:15:00.953 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:15:00.954 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:15:00.954 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:15:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:01.284 INFO:teuthology.orchestra.run.smithi148.stdout:generic/005 0s 2024-03-04T12:15:01.289 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/005 2024-03-04T12:15:01.289 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:15:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:01.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:15:01.328 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-03-04T12:15:01.502 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:15:01.506 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:15:01.506 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:15:01.507 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:15:01.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:21.862 INFO:teuthology.orchestra.run.smithi148.stdout:generic/006 20s 2024-03-04T12:15:21.867 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/006 2024-03-04T12:15:21.867 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:15:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:21.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:15:21.905 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-03-04T12:15:22.089 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:15:22.093 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:15:22.093 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:15:22.094 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:15:22.094 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:15:25.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:15:25.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:15:55.507 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:15:55.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:16:04.109 INFO:teuthology.orchestra.run.smithi148.stdout:generic/007 42s 2024-03-04T12:16:04.115 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/007 2024-03-04T12:16:04.115 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:04.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:04.237 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-03-04T12:16:04.424 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:04.428 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:04.428 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:04.429 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:04.429 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:04.703 INFO:teuthology.orchestra.run.smithi148.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-04T12:16:04.707 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/008 2024-03-04T12:16:04.707 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/008 2024-03-04T12:16:04.707 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:04.709 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:04.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:04.741 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-03-04T12:16:04.932 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:04.936 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:04.936 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:04.937 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:04.938 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:05.184 INFO:teuthology.orchestra.run.smithi148.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-04T12:16:05.187 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/009 2024-03-04T12:16:05.188 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/009 2024-03-04T12:16:05.188 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:05.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:05.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:05.203 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-03-04T12:16:05.397 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:05.401 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:05.401 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:05.402 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:06.012 INFO:teuthology.orchestra.run.smithi148.stdout:generic/010 0s 2024-03-04T12:16:06.082 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/010 2024-03-04T12:16:06.082 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:06.084 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:06.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:06.122 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-03-04T12:16:06.306 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:06.309 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:06.310 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:06.311 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:06.311 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:25.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:16:25.808 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:16:31.736 INFO:teuthology.orchestra.run.smithi148.stdout:generic/011 25s 2024-03-04T12:16:31.741 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/011 2024-03-04T12:16:31.741 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:31.744 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:31.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:31.787 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-03-04T12:16:31.961 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:31.965 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:31.965 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:31.966 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:32.293 INFO:teuthology.orchestra.run.smithi148.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:16:32.296 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/012 2024-03-04T12:16:32.297 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/012 2024-03-04T12:16:32.297 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:16:32.299 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:32.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:16:32.341 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-03-04T12:16:32.527 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:16:32.530 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:16:32.531 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:16:32.532 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:16:32.532 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:16:55.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:16:55.868 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:17:26.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:17:26.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:17:41.402 INFO:teuthology.orchestra.run.smithi148.stdout:generic/013 69s 2024-03-04T12:17:41.409 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/013 2024-03-04T12:17:41.409 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:41.412 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:41.435 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:41.435 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-03-04T12:17:41.678 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:41.684 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:41.685 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:41.686 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:42.091 INFO:teuthology.orchestra.run.smithi148.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:17:42.096 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/014 2024-03-04T12:17:42.096 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/014 2024-03-04T12:17:42.096 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:42.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:42.121 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-03-04T12:17:42.364 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:42.368 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:42.369 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:42.370 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:42.639 INFO:teuthology.orchestra.run.smithi148.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:17:42.645 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/015 2024-03-04T12:17:42.645 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/015 2024-03-04T12:17:42.645 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:42.648 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:42.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:42.664 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-03-04T12:17:42.915 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:42.920 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:42.921 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:42.922 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:43.303 INFO:teuthology.orchestra.run.smithi148.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:17:43.308 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/016 2024-03-04T12:17:43.308 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/016 2024-03-04T12:17:43.308 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:43.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:43.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:43.343 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-03-04T12:17:43.629 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:43.634 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:43.635 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:43.636 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:43.636 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:43.923 INFO:teuthology.orchestra.run.smithi148.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:17:43.928 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/017 2024-03-04T12:17:43.928 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/017 2024-03-04T12:17:43.928 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:43.931 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:43.950 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:43.950 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-03-04T12:17:44.188 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:44.193 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:44.194 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:44.195 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:44.195 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:44.599 INFO:teuthology.orchestra.run.smithi148.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2024-03-04T12:17:44.609 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/018 2024-03-04T12:17:44.609 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/018 2024-03-04T12:17:44.609 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:44.612 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:44.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:44.626 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-03-04T12:17:44.829 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:44.834 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:44.835 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:44.836 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:44.836 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:45.097 INFO:teuthology.orchestra.run.smithi148.stdout:generic/019 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:17:45.101 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/019 2024-03-04T12:17:45.101 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/019 2024-03-04T12:17:45.101 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:45.104 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:45.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:45.118 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-03-04T12:17:45.346 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:45.351 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:45.351 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:45.352 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:45.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:48.722 INFO:teuthology.orchestra.run.smithi148.stdout:generic/020 3s 2024-03-04T12:17:48.767 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/020 2024-03-04T12:17:48.767 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:48.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:48.803 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-03-04T12:17:49.098 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:49.102 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:49.103 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:49.104 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:49.104 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:49.490 INFO:teuthology.orchestra.run.smithi148.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:17:49.495 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/021 2024-03-04T12:17:49.495 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/021 2024-03-04T12:17:49.495 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:49.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:49.530 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-03-04T12:17:49.820 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:49.824 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:49.825 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:49.827 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:50.217 INFO:teuthology.orchestra.run.smithi148.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:17:50.222 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/022 2024-03-04T12:17:50.222 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/022 2024-03-04T12:17:50.222 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:50.226 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:50.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:50.254 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-03-04T12:17:50.541 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:50.546 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:50.546 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:50.547 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:51.755 INFO:teuthology.orchestra.run.smithi148.stdout:generic/023 1s 2024-03-04T12:17:51.835 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/023 2024-03-04T12:17:51.836 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:51.837 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:51.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:51.923 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-03-04T12:17:52.201 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:52.207 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:52.207 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:52.208 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:52.506 INFO:teuthology.orchestra.run.smithi148.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-03-04T12:17:52.511 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/024 2024-03-04T12:17:52.511 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/024 2024-03-04T12:17:52.511 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:52.514 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:52.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:52.543 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-03-04T12:17:52.833 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:52.839 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:52.839 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:52.841 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:52.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:53.147 INFO:teuthology.orchestra.run.smithi148.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-03-04T12:17:53.152 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/025 2024-03-04T12:17:53.152 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/025 2024-03-04T12:17:53.153 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:53.156 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:53.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:53.191 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-03-04T12:17:53.494 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:53.499 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:53.500 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:53.501 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:53.796 INFO:teuthology.orchestra.run.smithi148.stdout:generic/026 [not run] ceph does not define maximum ACL count 2024-03-04T12:17:53.801 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/026 2024-03-04T12:17:53.801 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/026 2024-03-04T12:17:53.801 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:53.805 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:53.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:53.831 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-03-04T12:17:54.074 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:54.080 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:54.080 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:54.081 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:54.081 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:54.343 INFO:teuthology.orchestra.run.smithi148.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:17:54.348 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/027 2024-03-04T12:17:54.348 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/027 2024-03-04T12:17:54.348 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:54.351 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:54.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:54.365 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-03-04T12:17:54.605 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:17:54.611 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:17:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:17:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:17:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:56.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:17:56.619 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:17:59.945 INFO:teuthology.orchestra.run.smithi148.stdout:generic/028 5s 2024-03-04T12:17:59.953 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/028 2024-03-04T12:17:59.953 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:17:59.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:17:59.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:17:59.974 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-03-04T12:18:00.218 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:00.223 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:00.224 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:00.226 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:00.226 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:03.087 INFO:teuthology.orchestra.run.smithi148.stdout:generic/029 3s 2024-03-04T12:18:03.094 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/029 2024-03-04T12:18:03.094 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:03.097 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:03.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:03.107 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-03-04T12:18:03.349 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:03.353 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:03.354 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:03.355 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:03.355 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:04.600 INFO:teuthology.orchestra.run.smithi148.stdout:generic/030 1s 2024-03-04T12:18:04.607 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/030 2024-03-04T12:18:04.607 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:04.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:04.622 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-03-04T12:18:04.861 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:04.865 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:04.866 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:04.867 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:04.867 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:05.210 INFO:teuthology.orchestra.run.smithi148.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-03-04T12:18:05.215 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/031 2024-03-04T12:18:05.215 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/031 2024-03-04T12:18:05.215 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:05.218 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:05.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:05.240 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-03-04T12:18:05.471 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:05.476 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:05.477 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:05.478 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:05.478 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:05.757 INFO:teuthology.orchestra.run.smithi148.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:05.762 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/032 2024-03-04T12:18:05.762 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/032 2024-03-04T12:18:05.763 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:05.765 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:05.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:05.786 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-03-04T12:18:06.022 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:06.029 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:06.029 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:06.031 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:06.031 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:06.384 INFO:teuthology.orchestra.run.smithi148.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-04T12:18:06.390 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/033 2024-03-04T12:18:06.390 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/033 2024-03-04T12:18:06.390 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:06.392 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:06.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:06.412 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-03-04T12:18:06.646 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:06.651 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:06.651 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:06.652 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:06.913 INFO:teuthology.orchestra.run.smithi148.stdout:generic/034 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:06.918 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/034 2024-03-04T12:18:06.918 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/034 2024-03-04T12:18:06.918 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:06.921 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:06.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:06.941 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-03-04T12:18:07.193 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:07.198 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:07.198 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:07.199 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:07.538 INFO:teuthology.orchestra.run.smithi148.stdout:generic/035 0s 2024-03-04T12:18:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/035 2024-03-04T12:18:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:07.550 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:07.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:07.578 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-03-04T12:18:07.867 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:07.874 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:07.875 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:07.876 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:18.232 INFO:teuthology.orchestra.run.smithi148.stdout:generic/036 11s 2024-03-04T12:18:18.240 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/036 2024-03-04T12:18:18.240 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:18.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:18.277 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-03-04T12:18:18.575 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:18.579 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:18.579 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:18.582 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:18.582 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:23.198 INFO:teuthology.orchestra.run.smithi148.stdout:generic/037 5s 2024-03-04T12:18:23.206 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/037 2024-03-04T12:18:23.206 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:23.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:23.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:23.219 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-03-04T12:18:23.462 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:23.469 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:23.749 INFO:teuthology.orchestra.run.smithi148.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:23.753 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/038 2024-03-04T12:18:23.754 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/038 2024-03-04T12:18:23.754 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:23.756 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:23.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:23.776 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-03-04T12:18:24.009 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:24.014 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:24.016 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:24.017 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:24.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:24.315 INFO:teuthology.orchestra.run.smithi148.stdout:generic/039 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:24.320 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/039 2024-03-04T12:18:24.320 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/039 2024-03-04T12:18:24.320 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:24.323 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:24.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:24.350 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-03-04T12:18:24.575 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:24.580 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:24.581 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:24.582 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:24.582 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:24.877 INFO:teuthology.orchestra.run.smithi148.stdout:generic/040 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:24.882 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/040 2024-03-04T12:18:24.882 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/040 2024-03-04T12:18:24.882 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:24.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:24.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:24.907 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-03-04T12:18:25.139 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:25.144 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:25.144 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:25.145 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:25.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:25.426 INFO:teuthology.orchestra.run.smithi148.stdout:generic/041 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:25.430 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/041 2024-03-04T12:18:25.430 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/041 2024-03-04T12:18:25.430 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:25.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:25.453 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-03-04T12:18:25.698 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:25.703 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:25.704 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:25.705 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:25.705 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:generic/042 [not run] ceph does not support shutdown 2024-03-04T12:18:26.047 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/042 2024-03-04T12:18:26.048 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/042 2024-03-04T12:18:26.048 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:26.050 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:26.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:26.069 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-03-04T12:18:26.305 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:26.310 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:26.310 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:26.312 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:26.645 INFO:teuthology.orchestra.run.smithi148.stdout:generic/043 [not run] ceph does not support shutdown 2024-03-04T12:18:26.653 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/043 2024-03-04T12:18:26.653 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/043 2024-03-04T12:18:26.653 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:26.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:26.671 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-03-04T12:18:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:26.910 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:27.169 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:18:27.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:18:27.269 INFO:teuthology.orchestra.run.smithi148.stdout:generic/044 [not run] ceph does not support shutdown 2024-03-04T12:18:27.278 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/044 2024-03-04T12:18:27.279 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/044 2024-03-04T12:18:27.279 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:27.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:27.297 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-03-04T12:18:27.573 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:27.578 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:27.579 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:27.580 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:27.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:27.927 INFO:teuthology.orchestra.run.smithi148.stdout:generic/045 [not run] ceph does not support shutdown 2024-03-04T12:18:27.935 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/045 2024-03-04T12:18:27.935 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/045 2024-03-04T12:18:27.935 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:27.938 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:27.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:27.955 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-03-04T12:18:28.197 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:28.202 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:28.202 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:28.204 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:28.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:28.552 INFO:teuthology.orchestra.run.smithi148.stdout:generic/046 [not run] ceph does not support shutdown 2024-03-04T12:18:28.560 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/046 2024-03-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/046 2024-03-04T12:18:28.561 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:28.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:28.580 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-03-04T12:18:28.813 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:28.817 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:28.818 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:28.819 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:28.819 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:29.161 INFO:teuthology.orchestra.run.smithi148.stdout:generic/047 [not run] ceph does not support shutdown 2024-03-04T12:18:29.177 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/047 2024-03-04T12:18:29.178 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/047 2024-03-04T12:18:29.178 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:29.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:29.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:29.196 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-03-04T12:18:29.442 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:29.447 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:29.448 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:29.449 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:29.449 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:29.795 INFO:teuthology.orchestra.run.smithi148.stdout:generic/048 [not run] ceph does not support shutdown 2024-03-04T12:18:29.817 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/048 2024-03-04T12:18:29.817 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/048 2024-03-04T12:18:29.817 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:29.819 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:29.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:29.835 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-03-04T12:18:30.068 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:30.074 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:30.074 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:30.075 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:30.076 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:30.426 INFO:teuthology.orchestra.run.smithi148.stdout:generic/049 [not run] ceph does not support shutdown 2024-03-04T12:18:30.436 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/049 2024-03-04T12:18:30.436 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/049 2024-03-04T12:18:30.436 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:30.439 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:30.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:30.457 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-03-04T12:18:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:30.705 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:30.707 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:30.708 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:30.709 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:31.058 INFO:teuthology.orchestra.run.smithi148.stdout:generic/050 [not run] ceph does not support shutdown 2024-03-04T12:18:31.067 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/050 2024-03-04T12:18:31.067 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/050 2024-03-04T12:18:31.067 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:31.070 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:31.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:31.084 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-03-04T12:18:31.319 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:31.325 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:31.326 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:31.327 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:31.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:31.685 INFO:teuthology.orchestra.run.smithi148.stdout:generic/051 [not run] ceph does not support shutdown 2024-03-04T12:18:31.690 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/051 2024-03-04T12:18:31.690 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/051 2024-03-04T12:18:31.691 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:31.693 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:31.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:31.708 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-03-04T12:18:31.943 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:31.949 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:31.950 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:31.951 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:32.294 INFO:teuthology.orchestra.run.smithi148.stdout:generic/052 [not run] ceph does not support shutdown 2024-03-04T12:18:32.304 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/052 2024-03-04T12:18:32.305 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/052 2024-03-04T12:18:32.305 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:32.308 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:32.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:32.322 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-03-04T12:18:32.561 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:32.566 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:32.566 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:32.567 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:32.567 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:33.178 INFO:teuthology.orchestra.run.smithi148.stdout:generic/053 1s 2024-03-04T12:18:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/053 2024-03-04T12:18:33.221 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:33.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:33.236 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-03-04T12:18:33.476 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:33.481 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:33.481 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:33.482 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:33.482 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:33.866 INFO:teuthology.orchestra.run.smithi148.stdout:generic/054 [not run] ceph does not support shutdown 2024-03-04T12:18:33.887 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/054 2024-03-04T12:18:33.887 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/054 2024-03-04T12:18:33.888 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:33.890 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:33.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:33.903 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-03-04T12:18:34.139 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:34.143 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:34.144 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:34.145 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:34.495 INFO:teuthology.orchestra.run.smithi148.stdout:generic/055 [not run] ceph does not support shutdown 2024-03-04T12:18:34.505 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/055 2024-03-04T12:18:34.505 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/055 2024-03-04T12:18:34.505 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:34.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:34.522 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-03-04T12:18:34.765 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:34.770 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:34.771 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:34.772 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:34.772 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:35.060 INFO:teuthology.orchestra.run.smithi148.stdout:generic/056 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:35.064 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/056 2024-03-04T12:18:35.064 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/056 2024-03-04T12:18:35.065 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:35.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:35.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:35.088 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-03-04T12:18:35.323 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:35.327 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:35.327 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:35.328 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:35.328 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:35.613 INFO:teuthology.orchestra.run.smithi148.stdout:generic/057 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:35.619 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/057 2024-03-04T12:18:35.619 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/057 2024-03-04T12:18:35.619 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:35.622 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:35.641 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:35.641 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-03-04T12:18:35.882 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:35.888 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:35.889 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:35.890 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:36.261 INFO:teuthology.orchestra.run.smithi148.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:36.265 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/058 2024-03-04T12:18:36.266 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/058 2024-03-04T12:18:36.266 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:36.269 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:36.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:36.310 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-03-04T12:18:36.620 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:36.626 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:36.626 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:36.627 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:36.628 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:36.897 INFO:teuthology.orchestra.run.smithi148.stdout:generic/059 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:36.902 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/059 2024-03-04T12:18:36.902 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/059 2024-03-04T12:18:36.902 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:36.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:36.920 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-03-04T12:18:37.161 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:37.167 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:37.167 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:37.169 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:37.169 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:37.641 INFO:teuthology.orchestra.run.smithi148.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:37.647 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/060 2024-03-04T12:18:37.647 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/060 2024-03-04T12:18:37.647 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:37.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:37.687 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-03-04T12:18:37.992 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:37.997 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:37.998 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:37.999 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:37.999 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:38.370 INFO:teuthology.orchestra.run.smithi148.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:38.375 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/061 2024-03-04T12:18:38.375 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/061 2024-03-04T12:18:38.375 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:38.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:38.413 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-03-04T12:18:38.712 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:38.718 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:38.718 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:38.719 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:38.719 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:40.290 INFO:teuthology.orchestra.run.smithi148.stdout:generic/062 2s 2024-03-04T12:18:40.300 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/062 2024-03-04T12:18:40.300 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:40.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:40.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:40.316 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-03-04T12:18:40.631 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:40.636 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:40.637 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:40.638 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:41.093 INFO:teuthology.orchestra.run.smithi148.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:18:41.098 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/063 2024-03-04T12:18:41.098 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/063 2024-03-04T12:18:41.098 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:41.101 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:41.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:41.136 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-03-04T12:18:41.455 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:41.460 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:41.460 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:41.462 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:41.462 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:18:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/064 2024-03-04T12:18:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/064 2024-03-04T12:18:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:41.753 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:41.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:41.775 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-03-04T12:18:42.006 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:42.011 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:42.011 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:42.013 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:42.013 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:42.300 INFO:teuthology.orchestra.run.smithi148.stdout:generic/065 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:42.304 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/065 2024-03-04T12:18:42.304 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/065 2024-03-04T12:18:42.304 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:42.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:42.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:42.327 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-03-04T12:18:42.570 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:42.575 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:42.576 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:42.578 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:42.578 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:42.847 INFO:teuthology.orchestra.run.smithi148.stdout:generic/066 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:42.852 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/066 2024-03-04T12:18:42.852 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/066 2024-03-04T12:18:42.852 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:42.855 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:42.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:42.869 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-03-04T12:18:43.124 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:43.128 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:43.128 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:43.130 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:43.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:43.407 INFO:teuthology.orchestra.run.smithi148.stdout:generic/067 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:18:43.412 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/067 2024-03-04T12:18:43.412 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/067 2024-03-04T12:18:43.412 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:43.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:43.434 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-03-04T12:18:43.664 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:43.669 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:43.670 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:43.671 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:43.671 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:43.944 INFO:teuthology.orchestra.run.smithi148.stdout:generic/068 [not run] ceph does not support freezing 2024-03-04T12:18:43.950 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/068 2024-03-04T12:18:43.950 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/068 2024-03-04T12:18:43.950 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:43.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:43.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:43.976 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-03-04T12:18:44.207 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:44.211 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:44.212 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:44.213 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:50.916 INFO:teuthology.orchestra.run.smithi148.stdout:generic/069 6s 2024-03-04T12:18:50.928 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/069 2024-03-04T12:18:50.928 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:18:50.930 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:50.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:18:50.943 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-03-04T12:18:51.239 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:18:51.245 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:18:51.246 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:18:51.247 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:18:51.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:18:57.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:18:57.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:19:15.611 INFO:teuthology.orchestra.run.smithi148.stdout:generic/070 24s 2024-03-04T12:19:15.617 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/070 2024-03-04T12:19:15.617 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:15.620 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:15.717 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:15.717 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-03-04T12:19:16.004 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:16.009 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:16.012 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:16.013 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:16.013 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:16.284 INFO:teuthology.orchestra.run.smithi148.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T12:19:16.289 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/071 2024-03-04T12:19:16.289 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/071 2024-03-04T12:19:16.289 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:16.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:16.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:16.309 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-03-04T12:19:16.545 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:16.550 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:16.551 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:16.552 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:16.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:16.932 INFO:teuthology.orchestra.run.smithi148.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-03-04T12:19:16.937 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/072 2024-03-04T12:19:16.938 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/072 2024-03-04T12:19:16.938 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:16.941 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:16.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:16.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-03-04T12:19:17.190 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:17.195 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:17.195 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:17.196 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:17.197 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:17.479 INFO:teuthology.orchestra.run.smithi148.stdout:generic/073 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:19:17.485 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/073 2024-03-04T12:19:17.485 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/073 2024-03-04T12:19:17.485 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:17.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:17.505 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-03-04T12:19:17.742 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:17.747 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:17.747 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:17.749 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:27.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:19:27.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:19:32.100 INFO:teuthology.orchestra.run.smithi148.stdout:generic/074 15s 2024-03-04T12:19:32.145 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/074 2024-03-04T12:19:32.145 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:33.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:33.298 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-03-04T12:19:33.600 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:33.606 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:33.607 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:33.608 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:33.608 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:53.844 INFO:teuthology.orchestra.run.smithi148.stdout:generic/075 20s 2024-03-04T12:19:53.848 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/075 2024-03-04T12:19:53.849 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:53.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:53.885 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-03-04T12:19:54.203 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:54.207 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:54.208 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:54.209 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:54.209 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:54.468 INFO:teuthology.orchestra.run.smithi148.stdout:generic/076 [not run] require 172.21.15.148:6789:/scratch to be local device 2024-03-04T12:19:54.470 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/076 2024-03-04T12:19:54.470 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/076 2024-03-04T12:19:54.470 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:54.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:54.488 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-03-04T12:19:54.721 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:54.726 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:54.726 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:54.728 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:54.728 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:56.073 INFO:teuthology.orchestra.run.smithi148.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:19:56.078 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/077 2024-03-04T12:19:56.078 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/077 2024-03-04T12:19:56.078 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:56.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:56.121 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-03-04T12:19:56.447 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:56.455 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:56.456 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:56.457 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:56.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:56.738 INFO:teuthology.orchestra.run.smithi148.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-03-04T12:19:56.742 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/078 2024-03-04T12:19:56.743 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/078 2024-03-04T12:19:56.744 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:56.746 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:56.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:56.783 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-03-04T12:19:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:57.104 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:57.104 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:57.105 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:57.386 INFO:teuthology.orchestra.run.smithi148.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-03-04T12:19:57.390 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/079 2024-03-04T12:19:57.390 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/079 2024-03-04T12:19:57.391 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:19:57.393 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:57.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:19:57.417 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-03-04T12:19:57.692 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:19:57.697 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:19:57.698 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:19:57.699 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:19:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:19:57.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:19:57.788 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:20:00.050 INFO:teuthology.orchestra.run.smithi148.stdout:generic/080 3s 2024-03-04T12:20:00.056 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/080 2024-03-04T12:20:00.056 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:00.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:00.081 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-03-04T12:20:00.318 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:00.324 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:00.324 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:00.635 INFO:teuthology.orchestra.run.smithi148.stdout:generic/081 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:20:00.641 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/081 2024-03-04T12:20:00.641 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/081 2024-03-04T12:20:00.641 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:00.644 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:00.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:00.658 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-03-04T12:20:00.919 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:00.924 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:00.924 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:00.927 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:01.199 INFO:teuthology.orchestra.run.smithi148.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:20:01.204 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/082 2024-03-04T12:20:01.204 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/082 2024-03-04T12:20:01.204 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:01.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:01.226 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:01.226 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-03-04T12:20:01.469 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:01.473 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:01.473 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:01.474 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:01.727 INFO:teuthology.orchestra.run.smithi148.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:20:01.732 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/083 2024-03-04T12:20:01.732 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/083 2024-03-04T12:20:01.732 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:01.735 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:01.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:01.750 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-03-04T12:20:01.989 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:01.994 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:01.994 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:01.995 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:07.562 INFO:teuthology.orchestra.run.smithi148.stdout:generic/084 5s 2024-03-04T12:20:07.699 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/084 2024-03-04T12:20:07.699 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:07.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:07.713 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-03-04T12:20:07.959 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:07.963 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:07.964 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:07.965 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:07.965 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:08.263 INFO:teuthology.orchestra.run.smithi148.stdout:generic/085 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:20:08.268 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/085 2024-03-04T12:20:08.268 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/085 2024-03-04T12:20:08.268 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:08.271 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:08.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:08.286 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-03-04T12:20:08.534 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:08.538 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:08.539 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:08.540 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:08.540 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:08.819 INFO:teuthology.orchestra.run.smithi148.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:20:08.823 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/086 2024-03-04T12:20:08.823 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/086 2024-03-04T12:20:08.823 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:08.848 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:08.848 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-03-04T12:20:09.081 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:09.086 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:09.087 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:09.088 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:09.088 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:09.448 INFO:teuthology.orchestra.run.smithi148.stdout:generic/087 0s 2024-03-04T12:20:09.456 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/087 2024-03-04T12:20:09.456 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:09.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:09.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:09.511 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-03-04T12:20:09.835 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:09.839 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:09.841 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:09.841 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:10.153 INFO:teuthology.orchestra.run.smithi148.stdout:generic/088 1s 2024-03-04T12:20:10.161 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/088 2024-03-04T12:20:10.161 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:20:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:10.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:20:10.196 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-03-04T12:20:10.521 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:20:10.527 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:20:10.528 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:20:10.530 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:20:10.530 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:20:27.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:20:27.954 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:20:58.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:20:58.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:20:58.197 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T12:20:58.192+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173583) UID: 0 2024-03-04T12:20:58.198 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T12:20:58.192+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173583) UID: 0 2024-03-04T12:24:19.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:24:19.660 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:24:50.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:24:50.003 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:25:20.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:25:20.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:25:20.479 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T12:25:20.472+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173852) UID: 0 2024-03-04T12:25:20.479 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T12:25:20.472+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173852) UID: 0 2024-03-04T12:28:24.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:28:24.081 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:28:24.719 INFO:teuthology.orchestra.run.smithi148.stdout:generic/089 494s 2024-03-04T12:28:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/089 2024-03-04T12:28:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:28:24.788 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:28:25.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:28:25.762 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-03-04T12:28:26.584 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:28:26.591 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:28:26.591 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:28:26.592 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:28:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:28:27.263 INFO:teuthology.orchestra.run.smithi148.stdout:generic/090 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:28:27.267 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/090 2024-03-04T12:28:27.267 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/090 2024-03-04T12:28:27.267 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:28:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:28:27.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:28:27.283 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-03-04T12:28:27.461 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:28:27.464 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:28:27.465 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:28:27.466 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:28:27.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:28:54.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:28:54.202 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:29:24.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:29:24.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:29:54.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:29:54.415 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:30:01.925 INFO:teuthology.orchestra.run.smithi148.stdout:generic/091 94s 2024-03-04T12:30:01.930 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/091 2024-03-04T12:30:01.930 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:01.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:01.977 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-03-04T12:30:02.355 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:02.358 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:02.358 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:02.360 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:02.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:02.587 INFO:teuthology.orchestra.run.smithi148.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:30:02.591 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/092 2024-03-04T12:30:02.591 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/092 2024-03-04T12:30:02.591 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:02.593 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:02.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:02.608 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-03-04T12:30:02.782 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:02.786 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:02.786 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:02.787 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:02.787 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:03.732 INFO:teuthology.orchestra.run.smithi148.stdout:generic/093 1s 2024-03-04T12:30:03.738 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/093 2024-03-04T12:30:03.738 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:03.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:03.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:03.792 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-03-04T12:30:04.172 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:04.176 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:04.176 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:04.177 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:04.431 INFO:teuthology.orchestra.run.smithi148.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:30:04.435 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/094 2024-03-04T12:30:04.435 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/094 2024-03-04T12:30:04.435 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:04.437 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:04.478 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:04.478 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-03-04T12:30:04.866 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:04.870 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:04.870 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:04.871 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:04.871 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout:generic/095 10s 2024-03-04T12:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/095 2024-03-04T12:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:14.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:14.706 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-03-04T12:30:14.936 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:14.940 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:14.940 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:14.941 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:15.221 INFO:teuthology.orchestra.run.smithi148.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-04T12:30:15.225 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/096 2024-03-04T12:30:15.225 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/096 2024-03-04T12:30:15.225 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:15.227 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:15.244 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:15.244 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-03-04T12:30:15.425 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:15.429 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:15.429 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:15.430 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:15.430 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:16.050 INFO:teuthology.orchestra.run.smithi148.stdout:generic/097 1s 2024-03-04T12:30:16.056 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/097 2024-03-04T12:30:16.056 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:16.058 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:16.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:16.074 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-03-04T12:30:16.246 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:16.250 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:16.250 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:16.251 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:16.251 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:generic/098 3s 2024-03-04T12:30:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/098 2024-03-04T12:30:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:19.593 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:19.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:19.601 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-03-04T12:30:19.781 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:19.785 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:19.785 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:19.786 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:20.580 INFO:teuthology.orchestra.run.smithi148.stdout:generic/099 1s 2024-03-04T12:30:20.585 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/099 2024-03-04T12:30:20.585 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:20.588 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:20.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:20.649 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-03-04T12:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:24.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:30:24.701 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:30:31.787 INFO:teuthology.orchestra.run.smithi148.stdout:generic/100 10s 2024-03-04T12:30:31.792 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/100 2024-03-04T12:30:31.792 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:31.794 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:34.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:34.623 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-03-04T12:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:35.178 INFO:teuthology.orchestra.run.smithi148.stdout:generic/101 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:30:35.182 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/101 2024-03-04T12:30:35.182 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/101 2024-03-04T12:30:35.182 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:35.184 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:35.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:35.195 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-03-04T12:30:35.364 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:35.368 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:35.368 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:35.369 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:35.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:35.597 INFO:teuthology.orchestra.run.smithi148.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:30:35.601 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/102 2024-03-04T12:30:35.601 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/102 2024-03-04T12:30:35.601 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:35.603 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:35.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:35.614 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-03-04T12:30:35.799 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:35.803 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:35.804 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:35.805 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:36.029 INFO:teuthology.orchestra.run.smithi148.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:30:36.033 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/103 2024-03-04T12:30:36.033 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/103 2024-03-04T12:30:36.033 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:36.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:36.079 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-03-04T12:30:36.456 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:36.460 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:36.460 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:36.461 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:36.678 INFO:teuthology.orchestra.run.smithi148.stdout:generic/104 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:30:36.682 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/104 2024-03-04T12:30:36.682 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/104 2024-03-04T12:30:36.682 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:36.684 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:36.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:36.697 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-03-04T12:30:36.875 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:36.879 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:36.879 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:36.880 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:36.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:37.286 INFO:teuthology.orchestra.run.smithi148.stdout:generic/105 1s 2024-03-04T12:30:37.296 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/105 2024-03-04T12:30:37.296 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:37.299 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:37.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:37.311 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-03-04T12:30:37.654 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:37.657 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:37.658 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:37.659 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:37.884 INFO:teuthology.orchestra.run.smithi148.stdout:generic/106 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:30:37.888 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/106 2024-03-04T12:30:37.888 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/106 2024-03-04T12:30:37.888 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:37.890 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:37.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:37.904 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-03-04T12:30:38.078 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:38.082 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:38.082 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:38.083 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:38.083 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:38.306 INFO:teuthology.orchestra.run.smithi148.stdout:generic/107 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:30:38.310 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/107 2024-03-04T12:30:38.310 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/107 2024-03-04T12:30:38.310 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:38.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:38.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:38.326 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-03-04T12:30:38.510 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:38.514 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:38.514 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:38.515 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:38.515 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout:generic/108 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/108 2024-03-04T12:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/108 2024-03-04T12:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:38.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:38.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-03-04T12:30:39.141 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:39.145 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:39.145 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:39.146 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:39.146 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:47.251 INFO:teuthology.orchestra.run.smithi148.stdout:generic/109 7s 2024-03-04T12:30:47.279 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/109 2024-03-04T12:30:47.279 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:30:47.281 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:47.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:30:47.290 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-03-04T12:30:47.741 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:30:47.745 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:30:47.745 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:30:47.746 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:30:47.746 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:30:54.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:30:54.902 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:30:54.954 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T12:30:54.947+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 200170) UID: 0 2024-03-04T12:30:54.954 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T12:30:54.947+0000 7fa25d993640 -1 received signal: Hangup from (PID: 200170) UID: 0 2024-03-04T12:31:04.996 INFO:teuthology.orchestra.run.smithi148.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:31:04.999 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/110 2024-03-04T12:31:04.999 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/110 2024-03-04T12:31:04.999 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:05.001 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:05.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:05.053 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-03-04T12:31:05.473 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:05.478 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:05.482 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:05.485 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:05.485 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:06.051 INFO:teuthology.orchestra.run.smithi148.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:31:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/111 2024-03-04T12:31:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/111 2024-03-04T12:31:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:06.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:06.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:06.094 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-03-04T12:31:06.471 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:06.477 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:06.477 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:06.478 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:06.479 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout:generic/112 16s 2024-03-04T12:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/112 2024-03-04T12:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:22.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:22.618 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-03-04T12:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:39.575 INFO:teuthology.orchestra.run.smithi148.stdout:generic/113 16s 2024-03-04T12:31:39.580 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/113 2024-03-04T12:31:39.580 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:39.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:39.606 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-03-04T12:31:39.980 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:39.984 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:39.984 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:39.985 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:40.240 INFO:teuthology.orchestra.run.smithi148.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:31:40.246 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/114 2024-03-04T12:31:40.246 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/114 2024-03-04T12:31:40.246 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:40.249 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:40.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:40.324 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-03-04T12:31:40.553 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:40.556 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:40.557 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:40.558 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:40.558 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:40.800 INFO:teuthology.orchestra.run.smithi148.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:31:40.804 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/115 2024-03-04T12:31:40.804 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/115 2024-03-04T12:31:40.804 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:40.806 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:40.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:40.865 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-03-04T12:31:41.257 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:41.263 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:41.264 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:41.268 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:41.268 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:41.516 INFO:teuthology.orchestra.run.smithi148.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:31:41.520 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/116 2024-03-04T12:31:41.520 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/116 2024-03-04T12:31:41.520 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:41.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:41.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:41.571 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-03-04T12:31:41.990 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:41.995 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:41.995 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:41.996 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:58.669 INFO:teuthology.orchestra.run.smithi148.stdout:generic/117 16s 2024-03-04T12:31:58.676 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/117 2024-03-04T12:31:58.676 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:58.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:58.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:58.690 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-03-04T12:31:58.870 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:58.876 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:58.876 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:58.876 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:58.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:59.473 INFO:teuthology.orchestra.run.smithi148.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:31:59.477 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/118 2024-03-04T12:31:59.477 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/118 2024-03-04T12:31:59.477 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:31:59.479 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:31:59.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:31:59.539 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-03-04T12:31:59.969 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:31:59.973 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:31:59.973 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:31:59.974 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:31:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:00.203 INFO:teuthology.orchestra.run.smithi148.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:32:00.207 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/119 2024-03-04T12:32:00.207 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/119 2024-03-04T12:32:00.207 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:00.210 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:00.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:00.314 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-03-04T12:32:00.788 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:00.794 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:00.794 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:00.795 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:00.796 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:00.992 INFO:teuthology.orchestra.run.smithi148.stdout:generic/120 [not run] atime not maintained by ceph 2024-03-04T12:32:00.995 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/120 2024-03-04T12:32:00.996 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/120 2024-03-04T12:32:00.996 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:01.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:01.009 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-03-04T12:32:01.178 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:01.181 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:01.182 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:01.182 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:01.182 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:01.415 INFO:teuthology.orchestra.run.smithi148.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:32:01.419 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/121 2024-03-04T12:32:01.419 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/121 2024-03-04T12:32:01.419 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:01.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:01.472 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-03-04T12:32:01.881 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:01.887 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:01.887 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:01.888 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:02.122 INFO:teuthology.orchestra.run.smithi148.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:32:02.126 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/122 2024-03-04T12:32:02.126 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/122 2024-03-04T12:32:02.127 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:02.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:02.182 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-03-04T12:32:02.616 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:02.620 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:02.620 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:02.621 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:02.621 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:03.118 INFO:teuthology.orchestra.run.smithi148.stdout:generic/123 1s 2024-03-04T12:32:03.123 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/123 2024-03-04T12:32:03.123 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:03.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:03.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:03.192 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-03-04T12:32:03.611 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:03.615 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:03.615 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:03.616 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:03.616 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:05.316 INFO:teuthology.orchestra.run.smithi148.stdout:generic/124 2s 2024-03-04T12:32:05.327 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/124 2024-03-04T12:32:05.327 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:32:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:32:05.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:32:05.823 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-03-04T12:32:06.298 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:32:06.309 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:32:06.310 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:32:06.312 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:32:06.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:33:07.796 INFO:teuthology.orchestra.run.smithi148.stdout:generic/125 61s 2024-03-04T12:33:07.801 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/125 2024-03-04T12:33:07.801 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:33:07.803 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:33:07.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:33:07.851 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-03-04T12:33:08.290 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:33:08.294 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:33:08.294 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:33:08.295 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:33:08.295 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:33:08.655 INFO:teuthology.orchestra.run.smithi148.stdout:generic/126 0s 2024-03-04T12:33:08.660 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/126 2024-03-04T12:33:08.660 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:33:08.663 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:33:08.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:33:08.737 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-03-04T12:33:09.164 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:33:09.168 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:33:09.168 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:33:09.169 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:33:09.169 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:33:44.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:33:44.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:34:14.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:34:14.643 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:34:44.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:34:44.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:35:14.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:35:14.925 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:35:44.995 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:35:44.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:36:15.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:36:15.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:36:45.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:36:45.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:37:15.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:37:15.330 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:37:45.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:37:45.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:38:15.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:38:15.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:38:45.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:38:45.626 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:39:15.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:39:15.921 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:39:46.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:39:46.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:40:16.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:40:16.122 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:40:46.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:40:46.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:41:16.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:41:16.556 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:41:46.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:41:46.653 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:41:47.914 INFO:teuthology.orchestra.run.smithi148.stdout:generic/127 518s 2024-03-04T12:41:47.919 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/127 2024-03-04T12:41:47.919 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:41:47.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:47.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:41:47.975 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-03-04T12:41:48.372 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:41:48.376 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:41:48.376 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:41:48.377 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:41:48.377 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:49.395 INFO:teuthology.orchestra.run.smithi148.stdout:generic/128 1s 2024-03-04T12:41:49.401 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/128 2024-03-04T12:41:49.401 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:41:49.403 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:49.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:41:49.412 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-03-04T12:41:49.592 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:41:49.596 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:41:49.596 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:41:49.597 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:41:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:49.818 INFO:teuthology.orchestra.run.smithi148.stdout:generic/129 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:41:49.821 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/129 2024-03-04T12:41:49.822 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/129 2024-03-04T12:41:49.822 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:41:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:49.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:41:49.845 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-03-04T12:41:50.021 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:41:50.025 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:41:50.025 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:41:50.026 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:41:50.026 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:50.266 INFO:teuthology.orchestra.run.smithi148.stdout:generic/130 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:41:50.270 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/130 2024-03-04T12:41:50.270 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/130 2024-03-04T12:41:50.270 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:41:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:50.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:41:50.294 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-03-04T12:41:50.477 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:41:50.481 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:41:50.481 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:41:50.482 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:41:50.482 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:51.825 INFO:teuthology.orchestra.run.smithi148.stdout:generic/131 1s 2024-03-04T12:41:51.830 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/131 2024-03-04T12:41:51.830 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:41:51.832 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:41:51.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:41:51.893 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-03-04T12:41:52.287 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:41:52.290 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:41:52.290 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:41:52.291 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:41:52.291 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:42:03.853 INFO:teuthology.orchestra.run.smithi148.stdout:generic/132 11s 2024-03-04T12:42:03.858 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/132 2024-03-04T12:42:03.858 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:42:03.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:42:03.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:42:03.869 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-03-04T12:42:04.049 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:42:04.053 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:42:04.053 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:42:04.054 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:42:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:42:16.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:42:16.788 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:42:46.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:42:46.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:43:17.342 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:43:17.345 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:43:47.450 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:43:47.457 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:44:17.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:44:17.523 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:44:47.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:44:47.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:45:18.014 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:45:18.017 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:45:48.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:45:48.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:45:59.700 INFO:teuthology.orchestra.run.smithi148.stdout:generic/133 235s 2024-03-04T12:45:59.706 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/133 2024-03-04T12:45:59.706 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:45:59.708 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:45:59.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:45:59.744 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-03-04T12:46:00.120 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:00.124 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:00.125 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:00.126 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:00.126 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:00.358 INFO:teuthology.orchestra.run.smithi148.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:00.362 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/134 2024-03-04T12:46:00.362 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/134 2024-03-04T12:46:00.362 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:00.365 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:00.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:00.417 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-03-04T12:46:00.897 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:00.902 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:00.903 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:00.904 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:00.904 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:01.845 INFO:teuthology.orchestra.run.smithi148.stdout:generic/135 0s 2024-03-04T12:46:01.851 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/135 2024-03-04T12:46:01.851 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:01.854 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:01.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:01.864 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-03-04T12:46:02.043 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:02.046 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:02.047 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:02.048 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:02.279 INFO:teuthology.orchestra.run.smithi148.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:46:02.283 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/136 2024-03-04T12:46:02.283 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/136 2024-03-04T12:46:02.284 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:02.285 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:02.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:02.357 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-03-04T12:46:02.741 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:02.745 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:02.745 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:02.746 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:02.746 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:02.982 INFO:teuthology.orchestra.run.smithi148.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:02.986 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/137 2024-03-04T12:46:02.986 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/137 2024-03-04T12:46:02.986 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:02.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:03.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:03.038 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-03-04T12:46:03.465 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:03.469 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:03.469 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:03.470 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:03.703 INFO:teuthology.orchestra.run.smithi148.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:03.706 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/138 2024-03-04T12:46:03.707 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/138 2024-03-04T12:46:03.707 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:03.709 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:03.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:03.757 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-03-04T12:46:04.179 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:04.182 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:04.183 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:04.184 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:04.184 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:04.412 INFO:teuthology.orchestra.run.smithi148.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/139 2024-03-04T12:46:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/139 2024-03-04T12:46:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:04.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:04.468 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-03-04T12:46:04.868 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:04.871 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:04.872 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:04.872 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:04.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:05.124 INFO:teuthology.orchestra.run.smithi148.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:05.128 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/140 2024-03-04T12:46:05.128 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/140 2024-03-04T12:46:05.128 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:05.131 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:05.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:05.193 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-03-04T12:46:05.589 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:05.593 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:05.593 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:05.594 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:05.595 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:05.933 INFO:teuthology.orchestra.run.smithi148.stdout:generic/141 0s 2024-03-04T12:46:05.939 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/141 2024-03-04T12:46:05.939 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:05.941 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:05.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:05.950 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-03-04T12:46:06.129 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:06.133 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:06.133 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:06.134 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:06.134 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:06.383 INFO:teuthology.orchestra.run.smithi148.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:06.384 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/142 2024-03-04T12:46:06.384 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/142 2024-03-04T12:46:06.384 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:06.386 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:06.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:06.432 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-03-04T12:46:06.834 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:06.838 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:06.838 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:06.839 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:06.839 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:07.066 INFO:teuthology.orchestra.run.smithi148.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:07.070 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/143 2024-03-04T12:46:07.070 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/143 2024-03-04T12:46:07.070 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:07.073 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:07.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:07.116 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-03-04T12:46:07.510 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:07.514 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:07.514 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:07.515 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:07.515 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:07.751 INFO:teuthology.orchestra.run.smithi148.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:07.755 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/144 2024-03-04T12:46:07.755 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/144 2024-03-04T12:46:07.755 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:07.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:07.804 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-03-04T12:46:08.189 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:08.193 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:08.193 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:08.194 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:08.437 INFO:teuthology.orchestra.run.smithi148.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:08.441 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/145 2024-03-04T12:46:08.441 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/145 2024-03-04T12:46:08.441 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:08.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:08.494 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-03-04T12:46:08.912 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:08.916 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:08.916 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:08.917 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:09.151 INFO:teuthology.orchestra.run.smithi148.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:09.154 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/146 2024-03-04T12:46:09.155 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/146 2024-03-04T12:46:09.155 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:09.157 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:09.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:09.219 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-03-04T12:46:09.636 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:09.639 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:09.640 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:09.641 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:09.641 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:09.876 INFO:teuthology.orchestra.run.smithi148.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:09.879 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/147 2024-03-04T12:46:09.880 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/147 2024-03-04T12:46:09.880 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:09.882 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:09.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:09.938 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-03-04T12:46:10.346 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:10.350 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:10.350 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:10.352 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:10.594 INFO:teuthology.orchestra.run.smithi148.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:10.597 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/148 2024-03-04T12:46:10.597 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/148 2024-03-04T12:46:10.597 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:10.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:10.653 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-03-04T12:46:11.045 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:11.049 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:11.049 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:11.050 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:11.050 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:11.299 INFO:teuthology.orchestra.run.smithi148.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:11.302 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/149 2024-03-04T12:46:11.303 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/149 2024-03-04T12:46:11.303 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:11.305 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:11.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:11.354 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-03-04T12:46:11.751 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:11.756 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:11.756 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:12.000 INFO:teuthology.orchestra.run.smithi148.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:12.003 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/150 2024-03-04T12:46:12.003 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/150 2024-03-04T12:46:12.003 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:12.006 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:12.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:12.057 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-03-04T12:46:12.461 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:12.465 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:12.465 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:12.466 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:12.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:12.696 INFO:teuthology.orchestra.run.smithi148.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:12.700 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/151 2024-03-04T12:46:12.700 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/151 2024-03-04T12:46:12.701 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:12.702 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:12.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:12.751 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-03-04T12:46:13.154 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:13.157 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:13.157 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:13.158 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:13.159 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:13.400 INFO:teuthology.orchestra.run.smithi148.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:13.404 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/152 2024-03-04T12:46:13.404 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/152 2024-03-04T12:46:13.404 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:13.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:13.459 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-03-04T12:46:13.870 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:13.873 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:13.874 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:13.875 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:14.109 INFO:teuthology.orchestra.run.smithi148.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:14.113 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/153 2024-03-04T12:46:14.113 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/153 2024-03-04T12:46:14.113 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:14.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:14.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:14.178 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-03-04T12:46:14.555 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:14.558 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:14.559 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:14.560 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:14.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:14.795 INFO:teuthology.orchestra.run.smithi148.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:14.799 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/154 2024-03-04T12:46:14.799 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/154 2024-03-04T12:46:14.799 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:14.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:14.859 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-03-04T12:46:15.243 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:15.247 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:15.247 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:15.248 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:15.487 INFO:teuthology.orchestra.run.smithi148.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:15.492 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/155 2024-03-04T12:46:15.492 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/155 2024-03-04T12:46:15.492 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:15.494 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:15.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:15.547 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-03-04T12:46:15.964 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:15.967 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:15.968 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:15.969 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:15.969 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:16.203 INFO:teuthology.orchestra.run.smithi148.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:16.207 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/156 2024-03-04T12:46:16.207 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/156 2024-03-04T12:46:16.208 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:16.210 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:16.260 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:16.260 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-03-04T12:46:16.660 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:16.663 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:16.664 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:16.665 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:16.665 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:16.902 INFO:teuthology.orchestra.run.smithi148.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:16.905 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/157 2024-03-04T12:46:16.905 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/157 2024-03-04T12:46:16.905 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:16.908 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:16.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:16.951 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-03-04T12:46:17.362 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:17.366 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:17.366 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:17.367 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:17.367 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:17.602 INFO:teuthology.orchestra.run.smithi148.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:46:17.606 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/158 2024-03-04T12:46:17.607 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/158 2024-03-04T12:46:17.607 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:17.608 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:17.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:17.655 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-03-04T12:46:18.071 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:18.075 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:18.075 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:18.077 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:18.077 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:18.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:46:18.225 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:46:18.321 INFO:teuthology.orchestra.run.smithi148.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-03-04T12:46:18.324 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/159 2024-03-04T12:46:18.325 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/159 2024-03-04T12:46:18.325 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:18.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:18.345 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-03-04T12:46:18.524 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:18.528 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:18.528 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:18.529 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:18.529 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:18.769 INFO:teuthology.orchestra.run.smithi148.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-03-04T12:46:18.772 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/160 2024-03-04T12:46:18.773 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/160 2024-03-04T12:46:18.773 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:18.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:18.794 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-03-04T12:46:18.979 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:18.983 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:18.983 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:18.984 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:18.984 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:19.322 INFO:teuthology.orchestra.run.smithi148.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:19.327 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/161 2024-03-04T12:46:19.327 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/161 2024-03-04T12:46:19.328 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:19.331 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:19.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:19.657 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-03-04T12:46:19.843 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:19.846 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:19.847 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:19.848 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:19.848 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:20.162 INFO:teuthology.orchestra.run.smithi148.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-03-04T12:46:20.199 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/162 2024-03-04T12:46:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/162 2024-03-04T12:46:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:20.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:20.544 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-03-04T12:46:20.728 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:20.732 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:20.733 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:20.733 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:21.041 INFO:teuthology.orchestra.run.smithi148.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-03-04T12:46:21.100 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/163 2024-03-04T12:46:21.100 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/163 2024-03-04T12:46:21.100 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:21.102 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:21.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:21.423 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-03-04T12:46:21.613 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:21.616 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:21.617 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:21.618 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:21.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:21.996 INFO:teuthology.orchestra.run.smithi148.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:22.002 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/164 2024-03-04T12:46:22.002 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/164 2024-03-04T12:46:22.002 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:22.006 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:22.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:22.295 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-03-04T12:46:22.477 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:22.481 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:22.481 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:22.482 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:22.482 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:22.823 INFO:teuthology.orchestra.run.smithi148.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:22.828 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/165 2024-03-04T12:46:22.828 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/165 2024-03-04T12:46:22.828 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:22.832 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:23.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:23.159 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-03-04T12:46:23.337 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:23.340 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:23.341 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:23.342 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:23.342 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:23.707 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/166 2024-03-04T12:46:23.708 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/166 2024-03-04T12:46:23.708 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:23.710 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:24.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:24.020 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-03-04T12:46:24.202 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:24.205 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:24.205 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:24.206 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:24.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:24.556 INFO:teuthology.orchestra.run.smithi148.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:24.563 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/167 2024-03-04T12:46:24.563 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/167 2024-03-04T12:46:24.563 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:24.565 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:24.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:24.895 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-03-04T12:46:25.067 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:25.071 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:25.071 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:25.072 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:25.072 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:25.415 INFO:teuthology.orchestra.run.smithi148.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:25.420 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/168 2024-03-04T12:46:25.421 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/168 2024-03-04T12:46:25.421 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:25.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:25.750 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-03-04T12:46:25.925 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:25.929 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:25.929 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:25.930 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:26.347 INFO:teuthology.orchestra.run.smithi148.stdout:generic/169 0s 2024-03-04T12:46:26.352 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/169 2024-03-04T12:46:26.353 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:26.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:26.363 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-03-04T12:46:26.537 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:26.541 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:26.541 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:26.542 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/170 2024-03-04T12:46:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/170 2024-03-04T12:46:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:27.131 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:27.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:27.438 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-03-04T12:46:27.613 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:27.617 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:27.617 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:27.618 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:27.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:27.993 INFO:teuthology.orchestra.run.smithi148.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:27.998 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/171 2024-03-04T12:46:27.998 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/171 2024-03-04T12:46:27.998 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:28.002 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:28.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:28.309 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-03-04T12:46:28.493 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:28.497 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:28.497 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:28.498 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:28.845 INFO:teuthology.orchestra.run.smithi148.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:28.852 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/172 2024-03-04T12:46:28.852 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/172 2024-03-04T12:46:28.852 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:28.854 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:29.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:29.194 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-03-04T12:46:29.372 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:29.376 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:29.376 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:29.377 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:29.377 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:29.746 INFO:teuthology.orchestra.run.smithi148.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:29.751 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/173 2024-03-04T12:46:29.751 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/173 2024-03-04T12:46:29.751 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:29.755 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:30.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:30.076 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-03-04T12:46:30.268 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:30.272 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:30.273 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:30.273 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:30.274 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:30.665 INFO:teuthology.orchestra.run.smithi148.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:30.672 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/174 2024-03-04T12:46:30.672 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/174 2024-03-04T12:46:30.672 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:30.676 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:30.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:30.962 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-03-04T12:46:31.142 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:31.146 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:31.146 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:31.147 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:31.504 INFO:teuthology.orchestra.run.smithi148.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:31.511 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/175 2024-03-04T12:46:31.511 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/175 2024-03-04T12:46:31.511 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:31.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:31.820 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-03-04T12:46:31.998 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:32.002 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:32.002 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:32.003 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:32.353 INFO:teuthology.orchestra.run.smithi148.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:32.362 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/176 2024-03-04T12:46:32.362 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/176 2024-03-04T12:46:32.362 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:32.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:32.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:32.686 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-03-04T12:46:32.869 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:32.873 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:32.873 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:32.874 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:33.211 INFO:teuthology.orchestra.run.smithi148.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:46:33.214 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/177 2024-03-04T12:46:33.214 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/177 2024-03-04T12:46:33.214 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:33.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:33.292 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-03-04T12:46:33.717 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:33.721 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:33.721 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:33.722 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:33.722 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:33.964 INFO:teuthology.orchestra.run.smithi148.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:33.968 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/178 2024-03-04T12:46:33.968 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/178 2024-03-04T12:46:33.968 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:33.970 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:34.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:34.026 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-03-04T12:46:34.440 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:34.444 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:34.444 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:34.445 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:34.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:34.674 INFO:teuthology.orchestra.run.smithi148.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:34.678 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/179 2024-03-04T12:46:34.678 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/179 2024-03-04T12:46:34.678 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:34.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:34.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:34.733 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-03-04T12:46:35.140 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:35.144 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:35.144 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:35.145 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:35.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:35.380 INFO:teuthology.orchestra.run.smithi148.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:35.383 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/180 2024-03-04T12:46:35.383 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/180 2024-03-04T12:46:35.384 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:35.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:35.427 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-03-04T12:46:35.847 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:35.851 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:35.851 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:35.852 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:36.085 INFO:teuthology.orchestra.run.smithi148.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:46:36.088 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/181 2024-03-04T12:46:36.089 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/181 2024-03-04T12:46:36.089 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:36.091 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:36.140 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:36.140 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-03-04T12:46:36.557 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:36.560 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:36.560 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:36.561 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:36.562 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:36.804 INFO:teuthology.orchestra.run.smithi148.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:46:36.807 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/182 2024-03-04T12:46:36.807 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/182 2024-03-04T12:46:36.808 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:36.810 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:36.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:36.870 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-03-04T12:46:37.291 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:37.295 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:37.295 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:37.296 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:37.296 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:37.639 INFO:teuthology.orchestra.run.smithi148.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:37.645 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/183 2024-03-04T12:46:37.646 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/183 2024-03-04T12:46:37.646 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:37.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:37.995 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-03-04T12:46:38.173 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:38.177 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:38.177 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:38.178 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:38.178 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:38.670 INFO:teuthology.orchestra.run.smithi148.stdout:generic/184 0s 2024-03-04T12:46:38.676 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/184 2024-03-04T12:46:38.676 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:38.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:38.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:38.697 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-03-04T12:46:38.874 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:38.878 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:38.878 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:38.879 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:39.249 INFO:teuthology.orchestra.run.smithi148.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:39.256 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/185 2024-03-04T12:46:39.256 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/185 2024-03-04T12:46:39.256 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:39.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:39.556 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-03-04T12:46:39.742 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:39.746 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:39.746 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:39.747 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:39.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:40.091 INFO:teuthology.orchestra.run.smithi148.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:40.098 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/186 2024-03-04T12:46:40.098 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/186 2024-03-04T12:46:40.098 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:40.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:40.407 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-03-04T12:46:40.592 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:40.596 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:40.596 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:40.597 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:40.597 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:40.935 INFO:teuthology.orchestra.run.smithi148.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:40.942 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/187 2024-03-04T12:46:40.942 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/187 2024-03-04T12:46:40.942 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:40.944 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:41.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:41.287 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-03-04T12:46:41.460 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:41.463 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:41.464 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:41.465 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:41.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:41.810 INFO:teuthology.orchestra.run.smithi148.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:41.816 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/188 2024-03-04T12:46:41.816 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/188 2024-03-04T12:46:41.816 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:41.819 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:42.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:42.149 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-03-04T12:46:42.323 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:42.327 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:42.327 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:42.328 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:42.328 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:42.675 INFO:teuthology.orchestra.run.smithi148.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:42.680 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/189 2024-03-04T12:46:42.680 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/189 2024-03-04T12:46:42.681 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:42.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:43.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:43.025 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-03-04T12:46:43.202 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:43.206 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:43.206 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:43.207 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:43.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:43.558 INFO:teuthology.orchestra.run.smithi148.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:43.564 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/190 2024-03-04T12:46:43.564 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/190 2024-03-04T12:46:43.564 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:43.568 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:43.866 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:43.866 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-03-04T12:46:44.059 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:44.061 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:44.062 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:44.063 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:44.063 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:44.426 INFO:teuthology.orchestra.run.smithi148.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:44.434 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/191 2024-03-04T12:46:44.434 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/191 2024-03-04T12:46:44.434 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:44.438 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:44.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:44.767 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-03-04T12:46:44.940 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:44.944 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:44.944 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:44.945 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:45.164 INFO:teuthology.orchestra.run.smithi148.stdout:generic/192 [not run] atime not maintained by ceph 2024-03-04T12:46:45.168 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/192 2024-03-04T12:46:45.168 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/192 2024-03-04T12:46:45.168 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:45.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:45.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:45.183 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-03-04T12:46:45.378 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:45.382 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:45.382 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:45.383 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:45.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:46.289 INFO:teuthology.orchestra.run.smithi148.stdout:generic/193 1s 2024-03-04T12:46:46.294 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/193 2024-03-04T12:46:46.294 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:46.296 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:46.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:46.370 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-03-04T12:46:46.787 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:46.789 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:46.790 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:46.790 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:47.135 INFO:teuthology.orchestra.run.smithi148.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:47.142 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/194 2024-03-04T12:46:47.142 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/194 2024-03-04T12:46:47.142 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:47.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:47.473 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-03-04T12:46:47.653 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:47.657 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:47.657 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:47.658 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:47.658 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:47.994 INFO:teuthology.orchestra.run.smithi148.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:48.000 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/195 2024-03-04T12:46:48.000 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/195 2024-03-04T12:46:48.000 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:48.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:48.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:46:48.305 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:46:48.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:48.356 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-03-04T12:46:48.530 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:48.535 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:48.535 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:48.536 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:48.536 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:48.958 INFO:teuthology.orchestra.run.smithi148.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:48.965 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/196 2024-03-04T12:46:48.965 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/196 2024-03-04T12:46:48.965 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:48.968 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:49.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:49.336 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-03-04T12:46:49.517 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:49.520 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:49.521 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:49.522 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:49.887 INFO:teuthology.orchestra.run.smithi148.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:49.893 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/197 2024-03-04T12:46:49.893 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/197 2024-03-04T12:46:49.893 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:49.897 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:50.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:50.280 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-03-04T12:46:50.452 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:50.456 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:50.457 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:50.458 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:51.136 INFO:teuthology.orchestra.run.smithi148.stdout:generic/198 1s 2024-03-04T12:46:51.141 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/198 2024-03-04T12:46:51.141 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:51.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:51.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:51.190 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-03-04T12:46:51.633 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:51.638 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:51.638 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:51.639 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:52.014 INFO:teuthology.orchestra.run.smithi148.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:52.020 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/199 2024-03-04T12:46:52.020 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/199 2024-03-04T12:46:52.020 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:52.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:52.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:52.361 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-03-04T12:46:52.541 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:52.545 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:52.545 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:52.547 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:52.547 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:52.904 INFO:teuthology.orchestra.run.smithi148.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:52.911 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/200 2024-03-04T12:46:52.911 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/200 2024-03-04T12:46:52.911 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:53.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:53.250 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-03-04T12:46:53.425 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:53.429 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:53.429 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:53.430 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:53.778 INFO:teuthology.orchestra.run.smithi148.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:53.783 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/201 2024-03-04T12:46:53.783 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/201 2024-03-04T12:46:53.783 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:53.786 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:54.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:54.114 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-03-04T12:46:54.295 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:54.299 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:54.300 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:54.300 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:54.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:54.649 INFO:teuthology.orchestra.run.smithi148.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:54.655 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/202 2024-03-04T12:46:54.656 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/202 2024-03-04T12:46:54.656 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:54.658 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:55.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:55.005 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-03-04T12:46:55.187 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:55.192 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:55.534 INFO:teuthology.orchestra.run.smithi148.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:55.539 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/203 2024-03-04T12:46:55.540 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/203 2024-03-04T12:46:55.540 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:55.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:55.859 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-03-04T12:46:56.036 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:56.040 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:56.040 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:56.041 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:57.956 INFO:teuthology.orchestra.run.smithi148.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:46:57.961 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/204 2024-03-04T12:46:57.961 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/204 2024-03-04T12:46:57.961 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:57.964 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:57.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:57.979 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-03-04T12:46:58.179 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:58.180 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:58.180 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:58.181 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:58.182 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:58.532 INFO:teuthology.orchestra.run.smithi148.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:58.537 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/205 2024-03-04T12:46:58.537 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/205 2024-03-04T12:46:58.538 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:58.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:58.887 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-03-04T12:46:59.062 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:59.066 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:59.066 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:59.067 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:59.421 INFO:teuthology.orchestra.run.smithi148.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:46:59.428 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/206 2024-03-04T12:46:59.428 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/206 2024-03-04T12:46:59.429 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:46:59.431 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:46:59.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:46:59.767 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-03-04T12:46:59.948 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:46:59.952 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:46:59.952 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:46:59.953 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:46:59.954 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:47:05.277 INFO:teuthology.orchestra.run.smithi148.stdout:generic/207 5s 2024-03-04T12:47:05.282 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/207 2024-03-04T12:47:05.282 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:47:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:47:05.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:47:05.321 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-03-04T12:47:05.756 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:47:05.760 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:47:05.760 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:47:05.761 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:47:05.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:47:18.799 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:47:18.802 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:47:48.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:47:48.965 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:48:19.100 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:48:19.106 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:48:49.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:48:49.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:48:49.360 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T12:48:49.353+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48855) UID: 0 2024-03-04T12:48:49.360 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T12:48:49.353+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48855) UID: 0 2024-03-04T12:48:49.361 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T12:48:49.353+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48855) UID: 0 2024-03-04T12:48:49.361 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T12:48:49.353+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48855) UID: 0 2024-03-04T12:50:26.119 INFO:teuthology.orchestra.run.smithi148.stdout:generic/208 201s 2024-03-04T12:50:26.124 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/208 2024-03-04T12:50:26.124 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:50:26.126 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:50:26.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:50:26.170 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-03-04T12:50:26.589 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:50:26.592 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:50:26.593 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:50:26.594 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:50:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:00.196 INFO:teuthology.orchestra.run.smithi148.stdout:generic/209 34s 2024-03-04T12:51:00.202 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/209 2024-03-04T12:51:00.202 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:00.205 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:00.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:00.261 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-03-04T12:51:00.690 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:00.695 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:00.695 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:01.025 INFO:teuthology.orchestra.run.smithi148.stdout:generic/210 1s 2024-03-04T12:51:01.030 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/210 2024-03-04T12:51:01.030 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:01.033 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:01.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:01.076 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-03-04T12:51:01.528 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:01.531 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:01.532 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:01.533 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:01.533 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:01.910 INFO:teuthology.orchestra.run.smithi148.stdout:generic/211 0s 2024-03-04T12:51:01.916 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/211 2024-03-04T12:51:01.916 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:01.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:01.931 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-03-04T12:51:02.110 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:02.113 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:02.114 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:02.114 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:02.408 INFO:teuthology.orchestra.run.smithi148.stdout:generic/212 0s 2024-03-04T12:51:02.413 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/212 2024-03-04T12:51:02.413 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:02.415 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:02.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:02.433 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-03-04T12:51:02.609 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:02.613 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:02.613 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:02.614 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:02.614 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:51:02.836 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/213 2024-03-04T12:51:02.836 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/213 2024-03-04T12:51:02.837 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:02.839 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:02.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:02.852 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-03-04T12:51:03.041 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:03.045 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:03.045 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:03.046 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:03.046 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:03.277 INFO:teuthology.orchestra.run.smithi148.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:51:03.281 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/214 2024-03-04T12:51:03.281 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/214 2024-03-04T12:51:03.281 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:03.283 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:03.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:03.297 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-03-04T12:51:03.477 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:03.481 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:03.481 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:03.482 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:05.861 INFO:teuthology.orchestra.run.smithi148.stdout:generic/215 2s 2024-03-04T12:51:05.867 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/215 2024-03-04T12:51:05.867 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:05.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:05.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:05.899 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-03-04T12:51:06.080 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:06.083 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:06.084 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:06.085 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:06.085 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:06.708 INFO:teuthology.orchestra.run.smithi148.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:06.713 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/216 2024-03-04T12:51:06.714 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/216 2024-03-04T12:51:06.714 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:06.723 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:07.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:07.084 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-03-04T12:51:07.262 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:07.265 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:07.266 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:07.267 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:07.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:07.628 INFO:teuthology.orchestra.run.smithi148.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:07.634 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/217 2024-03-04T12:51:07.635 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/217 2024-03-04T12:51:07.635 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:07.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:07.995 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-03-04T12:51:08.180 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:08.183 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:08.184 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:08.185 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:08.553 INFO:teuthology.orchestra.run.smithi148.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:08.560 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/218 2024-03-04T12:51:08.560 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/218 2024-03-04T12:51:08.560 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:08.562 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:08.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:08.948 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-03-04T12:51:09.123 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:09.127 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:09.127 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:09.129 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:09.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:09.342 INFO:teuthology.orchestra.run.smithi148.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:09.346 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/219 2024-03-04T12:51:09.346 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/219 2024-03-04T12:51:09.346 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:09.349 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:09.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:09.362 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-03-04T12:51:09.539 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:09.543 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:09.543 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:09.544 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:09.908 INFO:teuthology.orchestra.run.smithi148.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:09.914 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/220 2024-03-04T12:51:09.914 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/220 2024-03-04T12:51:09.914 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:09.916 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:10.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:10.272 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-03-04T12:51:10.454 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:10.457 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:10.458 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:10.459 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:10.459 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:11.717 INFO:teuthology.orchestra.run.smithi148.stdout:generic/221 1s 2024-03-04T12:51:11.723 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/221 2024-03-04T12:51:11.723 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:11.726 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:11.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:11.748 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-03-04T12:51:11.927 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:11.931 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:11.932 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:11.933 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:11.933 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:12.302 INFO:teuthology.orchestra.run.smithi148.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:12.309 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/222 2024-03-04T12:51:12.309 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/222 2024-03-04T12:51:12.309 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:12.311 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:12.695 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:12.695 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-03-04T12:51:12.872 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:12.876 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:12.876 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:12.877 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:12.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:13.103 INFO:teuthology.orchestra.run.smithi148.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:51:13.107 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/223 2024-03-04T12:51:13.107 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/223 2024-03-04T12:51:13.107 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:13.109 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:13.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:13.130 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-03-04T12:51:13.323 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:13.326 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:13.327 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:13.327 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:13.328 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:13.543 INFO:teuthology.orchestra.run.smithi148.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:51:13.546 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/224 2024-03-04T12:51:13.547 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/224 2024-03-04T12:51:13.547 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:13.549 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:13.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:13.562 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-03-04T12:51:13.742 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:13.746 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:13.746 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:13.747 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:13.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:14.009 INFO:teuthology.orchestra.run.smithi148.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:51:14.012 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/225 2024-03-04T12:51:14.013 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/225 2024-03-04T12:51:14.013 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:14.015 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:14.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:14.072 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-03-04T12:51:14.472 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:14.476 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:14.476 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:14.477 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:14.477 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:14.723 INFO:teuthology.orchestra.run.smithi148.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:51:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/226 2024-03-04T12:51:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/226 2024-03-04T12:51:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:14.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:14.747 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-03-04T12:51:14.928 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:14.932 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:14.932 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:14.933 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:15.292 INFO:teuthology.orchestra.run.smithi148.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:15.299 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/227 2024-03-04T12:51:15.299 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/227 2024-03-04T12:51:15.299 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:15.644 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:15.644 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-03-04T12:51:15.823 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:15.826 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:15.827 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:15.828 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:15.828 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:16.069 INFO:teuthology.orchestra.run.smithi148.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:51:16.073 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/228 2024-03-04T12:51:16.073 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/228 2024-03-04T12:51:16.073 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:16.075 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:16.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:16.089 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-03-04T12:51:16.264 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:16.267 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:16.268 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:16.269 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:16.269 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:16.603 INFO:teuthology.orchestra.run.smithi148.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:16.612 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/229 2024-03-04T12:51:16.612 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/229 2024-03-04T12:51:16.612 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:16.616 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:16.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:16.965 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-03-04T12:51:17.143 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:17.147 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:17.147 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:17.148 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:17.148 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:17.363 INFO:teuthology.orchestra.run.smithi148.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:17.367 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/230 2024-03-04T12:51:17.367 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/230 2024-03-04T12:51:17.367 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:17.369 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:17.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:17.399 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-03-04T12:51:17.580 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:17.583 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:17.584 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:17.585 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:17.585 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:17.787 INFO:teuthology.orchestra.run.smithi148.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:17.791 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/231 2024-03-04T12:51:17.791 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/231 2024-03-04T12:51:17.791 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:17.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:17.810 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-03-04T12:51:17.992 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:17.995 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:17.996 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:17.997 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:18.207 INFO:teuthology.orchestra.run.smithi148.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:18.211 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/232 2024-03-04T12:51:18.211 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/232 2024-03-04T12:51:18.211 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:18.213 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:18.226 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:18.226 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-03-04T12:51:18.400 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:18.404 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:18.404 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:18.405 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:18.606 INFO:teuthology.orchestra.run.smithi148.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:18.610 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/233 2024-03-04T12:51:18.610 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/233 2024-03-04T12:51:18.610 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:18.613 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:18.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:18.625 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-03-04T12:51:18.810 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:18.814 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:18.815 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:18.816 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:18.816 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:19.024 INFO:teuthology.orchestra.run.smithi148.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:19.028 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/234 2024-03-04T12:51:19.028 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/234 2024-03-04T12:51:19.028 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:19.030 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:19.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:19.043 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-03-04T12:51:19.223 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:19.227 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:19.228 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:19.228 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:19.229 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:19.450 INFO:teuthology.orchestra.run.smithi148.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:51:19.454 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/235 2024-03-04T12:51:19.454 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/235 2024-03-04T12:51:19.454 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:19.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:19.485 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-03-04T12:51:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:19.666 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:20.941 INFO:teuthology.orchestra.run.smithi148.stdout:generic/236 1s 2024-03-04T12:51:20.947 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/236 2024-03-04T12:51:20.947 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:20.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:20.990 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-03-04T12:51:21.419 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:21.423 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:21.424 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:21.425 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:21.764 INFO:teuthology.orchestra.run.smithi148.stdout:generic/237 0s 2024-03-04T12:51:21.769 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/237 2024-03-04T12:51:21.770 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:21.772 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:21.828 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:21.828 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-03-04T12:51:22.244 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:22.248 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:22.248 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:22.249 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:22.249 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:22.620 INFO:teuthology.orchestra.run.smithi148.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:51:22.626 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/238 2024-03-04T12:51:22.627 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/238 2024-03-04T12:51:22.627 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:22.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:22.962 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-03-04T12:51:23.145 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:23.148 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:23.149 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:23.150 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:23.150 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:23.377 INFO:teuthology.orchestra.run.smithi148.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:51:23.381 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/239 2024-03-04T12:51:23.381 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/239 2024-03-04T12:51:23.381 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:23.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:23.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:23.418 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-03-04T12:51:23.597 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:23.602 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:23.602 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:23.603 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:23.820 INFO:teuthology.orchestra.run.smithi148.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-03-04T12:51:23.824 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/240 2024-03-04T12:51:23.824 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/240 2024-03-04T12:51:23.824 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:51:23.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:23.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:51:23.849 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-03-04T12:51:24.029 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:51:24.032 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:51:24.033 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:51:24.034 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:51:24.034 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:51:43.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:51:43.662 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:51:43.696 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:51:43.690+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 265316) UID: 0 2024-03-04T12:51:43.696 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:51:43.690+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 265316) UID: 0 2024-03-04T12:51:43.697 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:51:43.690+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 265316) UID: 0 2024-03-04T12:51:43.697 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:51:43.690+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 265316) UID: 0 2024-03-04T12:52:37.135 INFO:teuthology.orchestra.run.smithi148.stdout:generic/241 73s 2024-03-04T12:52:37.179 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/241 2024-03-04T12:52:37.179 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:37.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:37.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:37.564 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-03-04T12:52:38.023 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:38.026 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:38.027 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:38.028 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:38.028 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:38.660 INFO:teuthology.orchestra.run.smithi148.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:52:38.671 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/242 2024-03-04T12:52:38.671 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/242 2024-03-04T12:52:38.671 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:38.675 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:39.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:39.071 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-03-04T12:52:39.241 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:39.245 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:39.245 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:39.246 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:39.589 INFO:teuthology.orchestra.run.smithi148.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:52:39.594 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/243 2024-03-04T12:52:39.595 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/243 2024-03-04T12:52:39.595 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:39.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:39.998 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-03-04T12:52:40.174 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:40.177 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:40.178 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:40.179 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:40.382 INFO:teuthology.orchestra.run.smithi148.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:52:40.385 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/244 2024-03-04T12:52:40.386 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/244 2024-03-04T12:52:40.386 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:40.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:40.400 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-03-04T12:52:40.579 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:40.582 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:40.583 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:40.584 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:40.584 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:40.863 INFO:teuthology.orchestra.run.smithi148.stdout:generic/245 0s 2024-03-04T12:52:40.868 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/245 2024-03-04T12:52:40.868 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:40.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:40.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:40.954 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-03-04T12:52:41.407 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:41.411 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:41.412 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:41.412 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:41.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:41.700 INFO:teuthology.orchestra.run.smithi148.stdout:generic/246 0s 2024-03-04T12:52:41.705 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/246 2024-03-04T12:52:41.705 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:41.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:41.757 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-03-04T12:52:42.254 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:42.257 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:42.258 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:42.259 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:42.259 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:50.976 INFO:teuthology.orchestra.run.smithi148.stdout:generic/247 8s 2024-03-04T12:52:50.981 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/247 2024-03-04T12:52:50.981 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:50.983 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:50.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:50.993 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-03-04T12:52:51.168 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:51.171 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:51.172 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:51.176 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:51.176 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:51.467 INFO:teuthology.orchestra.run.smithi148.stdout:generic/248 0s 2024-03-04T12:52:51.472 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/248 2024-03-04T12:52:51.472 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:51.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:51.496 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-03-04T12:52:51.675 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:51.679 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:51.680 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:51.681 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:52.093 INFO:teuthology.orchestra.run.smithi148.stdout:generic/249 1s 2024-03-04T12:52:52.099 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/249 2024-03-04T12:52:52.099 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:52.101 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:52.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:52.152 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-03-04T12:52:52.640 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:52.643 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:52.644 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:52.645 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:52.900 INFO:teuthology.orchestra.run.smithi148.stdout:generic/250 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:52:52.904 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/250 2024-03-04T12:52:52.904 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/250 2024-03-04T12:52:52.904 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:52.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:52.919 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-03-04T12:52:53.099 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:53.102 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:53.103 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:53.104 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:53.394 INFO:teuthology.orchestra.run.smithi148.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:53.401 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/251 2024-03-04T12:52:53.401 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/251 2024-03-04T12:52:53.401 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:53.403 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:53.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:53.414 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-03-04T12:52:53.599 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:53.603 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:53.603 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:53.604 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:53.604 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:53.827 INFO:teuthology.orchestra.run.smithi148.stdout:generic/252 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:52:53.831 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/252 2024-03-04T12:52:53.831 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/252 2024-03-04T12:52:53.831 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:53.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:53.846 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-03-04T12:52:54.018 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:54.022 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:54.022 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:54.023 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:54.024 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:54.368 INFO:teuthology.orchestra.run.smithi148.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:52:54.385 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/253 2024-03-04T12:52:54.385 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/253 2024-03-04T12:52:54.385 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:54.765 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:54.765 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-03-04T12:52:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:54.948 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:55.313 INFO:teuthology.orchestra.run.smithi148.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:52:55.319 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/254 2024-03-04T12:52:55.320 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/254 2024-03-04T12:52:55.320 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:55.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:55.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:55.701 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-03-04T12:52:55.881 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:55.885 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:55.885 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:55.886 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:55.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:56.200 INFO:teuthology.orchestra.run.smithi148.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:52:56.205 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/255 2024-03-04T12:52:56.205 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/255 2024-03-04T12:52:56.205 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:56.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:56.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:56.242 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-03-04T12:52:56.730 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:56.734 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:56.734 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:56.735 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:56.735 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:57.733 INFO:teuthology.orchestra.run.smithi148.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:52:57.736 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/256 2024-03-04T12:52:57.737 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/256 2024-03-04T12:52:57.737 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:57.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:57.755 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-03-04T12:52:57.932 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:57.936 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:57.936 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:57.937 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:57.937 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:58.625 INFO:teuthology.orchestra.run.smithi148.stdout:generic/257 0s 2024-03-04T12:52:58.630 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/257 2024-03-04T12:52:58.630 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:58.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:58.680 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-03-04T12:52:59.010 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:59.014 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:59.014 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:59.015 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:59.316 INFO:teuthology.orchestra.run.smithi148.stdout:generic/258 [not run] ceph does not support negative timestamps 2024-03-04T12:52:59.322 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/258 2024-03-04T12:52:59.323 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/258 2024-03-04T12:52:59.323 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:59.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:59.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:52:59.357 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-03-04T12:52:59.539 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:52:59.542 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:52:59.542 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:52:59.543 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:52:59.544 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:52:59.898 INFO:teuthology.orchestra.run.smithi148.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:52:59.904 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/259 2024-03-04T12:52:59.904 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/259 2024-03-04T12:52:59.905 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:52:59.907 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:00.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:00.300 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-03-04T12:53:00.485 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:00.489 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:00.489 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:00.490 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:00.490 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:00.799 INFO:teuthology.orchestra.run.smithi148.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:00.807 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/260 2024-03-04T12:53:00.807 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/260 2024-03-04T12:53:00.807 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:00.809 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:00.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:00.821 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-03-04T12:53:00.996 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:01.000 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:01.000 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:01.001 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:01.352 INFO:teuthology.orchestra.run.smithi148.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:01.358 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/261 2024-03-04T12:53:01.358 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/261 2024-03-04T12:53:01.358 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:01.361 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:01.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:01.743 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-03-04T12:53:01.916 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:01.920 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:01.921 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:01.922 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:02.270 INFO:teuthology.orchestra.run.smithi148.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:02.277 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/262 2024-03-04T12:53:02.278 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/262 2024-03-04T12:53:02.278 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:02.281 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:02.646 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:02.647 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-03-04T12:53:02.833 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:02.836 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:02.836 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:02.838 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:02.838 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:41.260 INFO:teuthology.orchestra.run.smithi148.stdout:generic/263 39s 2024-03-04T12:53:41.265 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/263 2024-03-04T12:53:41.265 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:41.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:41.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:41.323 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-03-04T12:53:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:41.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:42.209 INFO:teuthology.orchestra.run.smithi148.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:42.214 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/264 2024-03-04T12:53:42.214 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/264 2024-03-04T12:53:42.214 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:42.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:42.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:42.609 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-03-04T12:53:42.787 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:42.790 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:42.790 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:42.791 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:42.791 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:43.216 INFO:teuthology.orchestra.run.smithi148.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:43.223 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/265 2024-03-04T12:53:43.223 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/265 2024-03-04T12:53:43.223 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:43.225 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:43.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:43.561 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-03-04T12:53:43.761 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:43.764 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:43.765 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:43.766 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:43.766 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:44.163 INFO:teuthology.orchestra.run.smithi148.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:44.170 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/266 2024-03-04T12:53:44.170 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/266 2024-03-04T12:53:44.170 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:44.172 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:44.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:44.554 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-03-04T12:53:44.739 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:44.743 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:44.743 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:44.744 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:44.744 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:45.123 INFO:teuthology.orchestra.run.smithi148.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:45.130 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/267 2024-03-04T12:53:45.130 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/267 2024-03-04T12:53:45.130 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:45.132 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:45.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:45.510 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-03-04T12:53:45.695 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:45.699 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:45.700 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:45.701 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:45.701 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:46.089 INFO:teuthology.orchestra.run.smithi148.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:46.099 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/268 2024-03-04T12:53:46.099 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/268 2024-03-04T12:53:46.099 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:46.101 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:46.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:46.461 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-03-04T12:53:46.636 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:46.640 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:46.640 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:46.641 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:46.641 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:46.850 INFO:teuthology.orchestra.run.smithi148.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:53:46.854 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/269 2024-03-04T12:53:46.854 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/269 2024-03-04T12:53:46.854 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:46.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:46.879 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-03-04T12:53:47.058 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:47.074 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:47.074 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:47.074 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:47.074 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:47.287 INFO:teuthology.orchestra.run.smithi148.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:53:47.290 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/270 2024-03-04T12:53:47.291 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/270 2024-03-04T12:53:47.291 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:47.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:47.315 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-03-04T12:53:47.486 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:47.490 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:47.490 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:47.491 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:47.872 INFO:teuthology.orchestra.run.smithi148.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:47.879 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/271 2024-03-04T12:53:47.879 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/271 2024-03-04T12:53:47.879 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:47.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:48.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:48.253 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-03-04T12:53:48.435 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:48.438 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:48.439 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:48.440 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:48.440 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:48.809 INFO:teuthology.orchestra.run.smithi148.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:48.817 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/272 2024-03-04T12:53:48.817 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/272 2024-03-04T12:53:48.817 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:48.819 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:49.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:49.145 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-03-04T12:53:49.323 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:49.326 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:49.328 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:49.551 INFO:teuthology.orchestra.run.smithi148.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:53:49.554 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/273 2024-03-04T12:53:49.555 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/273 2024-03-04T12:53:49.555 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:49.580 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:49.580 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-03-04T12:53:49.752 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:49.756 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:49.757 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:49.757 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:49.757 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:49.987 INFO:teuthology.orchestra.run.smithi148.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T12:53:49.991 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/274 2024-03-04T12:53:49.991 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/274 2024-03-04T12:53:49.991 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:49.993 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:50.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:50.009 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-03-04T12:53:50.194 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:50.198 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:50.198 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:50.199 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:50.433 INFO:teuthology.orchestra.run.smithi148.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:53:50.437 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/275 2024-03-04T12:53:50.437 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/275 2024-03-04T12:53:50.437 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:50.439 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:50.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:50.463 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-03-04T12:53:50.635 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:50.639 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:50.639 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:50.640 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:50.640 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:51.008 INFO:teuthology.orchestra.run.smithi148.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:51.013 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/276 2024-03-04T12:53:51.013 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/276 2024-03-04T12:53:51.013 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:51.016 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:51.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:51.325 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-03-04T12:53:51.505 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:51.509 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:51.509 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:51.510 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:51.848 INFO:teuthology.orchestra.run.smithi148.stdout:generic/277 [not run] file system doesn't support chattr +A 2024-03-04T12:53:51.852 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/277 2024-03-04T12:53:51.852 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/277 2024-03-04T12:53:51.852 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:51.855 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:51.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:51.873 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-03-04T12:53:52.048 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:52.052 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:52.052 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:52.053 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:52.053 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:52.436 INFO:teuthology.orchestra.run.smithi148.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:52.443 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/278 2024-03-04T12:53:52.443 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/278 2024-03-04T12:53:52.444 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:52.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:52.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:52.809 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-03-04T12:53:52.986 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:52.989 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:52.990 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:52.991 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:53.445 INFO:teuthology.orchestra.run.smithi148.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:53.452 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/279 2024-03-04T12:53:53.453 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/279 2024-03-04T12:53:53.453 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:53.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:53.844 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-03-04T12:53:54.021 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:54.025 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:54.025 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:54.026 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:54.026 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:54.273 INFO:teuthology.orchestra.run.smithi148.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:53:54.277 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/280 2024-03-04T12:53:54.277 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/280 2024-03-04T12:53:54.277 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:54.280 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:54.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:54.295 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-03-04T12:53:54.478 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:54.482 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:54.483 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:54.483 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/281 2024-03-04T12:53:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/281 2024-03-04T12:53:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:54.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:55.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:55.246 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-03-04T12:53:55.438 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:55.441 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:55.442 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:55.443 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:55.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:55.814 INFO:teuthology.orchestra.run.smithi148.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:55.824 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/282 2024-03-04T12:53:55.824 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/282 2024-03-04T12:53:55.824 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:55.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:56.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:56.177 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-03-04T12:53:56.355 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:56.359 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:56.359 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:56.360 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:56.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:56.743 INFO:teuthology.orchestra.run.smithi148.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:56.750 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/283 2024-03-04T12:53:56.751 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/283 2024-03-04T12:53:56.751 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:56.752 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:57.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:57.031 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-03-04T12:53:57.240 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:57.244 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:57.245 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:57.246 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:57.246 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:57.612 INFO:teuthology.orchestra.run.smithi148.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:53:57.618 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/284 2024-03-04T12:53:57.618 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/284 2024-03-04T12:53:57.618 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:57.621 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:57.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:57.988 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-03-04T12:53:58.166 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:53:58.170 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:53:58.170 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:53:58.171 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:53:58.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:59.332 INFO:teuthology.orchestra.run.smithi148.stdout:generic/285 1s 2024-03-04T12:53:59.337 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/285 2024-03-04T12:53:59.338 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:53:59.340 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:53:59.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:53:59.740 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-03-04T12:54:00.236 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:00.241 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:00.242 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:00.243 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:00.243 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:00.594 INFO:teuthology.orchestra.run.smithi148.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:54:00.599 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/286 2024-03-04T12:54:00.599 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/286 2024-03-04T12:54:00.599 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:00.601 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:00.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:00.625 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-03-04T12:54:00.900 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:00.904 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:00.905 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:00.906 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:00.906 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:01.371 INFO:teuthology.orchestra.run.smithi148.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:01.377 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/287 2024-03-04T12:54:01.378 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/287 2024-03-04T12:54:01.378 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:01.381 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:01.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:01.698 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-03-04T12:54:01.994 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:02.001 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:02.001 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:02.002 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:02.002 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:02.443 INFO:teuthology.orchestra.run.smithi148.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:02.460 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/288 2024-03-04T12:54:02.460 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/288 2024-03-04T12:54:02.460 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:02.463 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:02.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:02.483 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-03-04T12:54:02.743 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:02.747 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:02.747 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:02.748 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:02.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:03.253 INFO:teuthology.orchestra.run.smithi148.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:03.260 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/289 2024-03-04T12:54:03.260 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/289 2024-03-04T12:54:03.260 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:03.262 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:03.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:03.670 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-03-04T12:54:03.930 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:03.936 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:03.936 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:03.939 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:03.939 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:04.418 INFO:teuthology.orchestra.run.smithi148.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:04.429 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/290 2024-03-04T12:54:04.429 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/290 2024-03-04T12:54:04.429 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:04.432 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:04.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:04.837 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-03-04T12:54:05.098 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:05.105 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:05.106 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:05.107 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:05.107 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:05.573 INFO:teuthology.orchestra.run.smithi148.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:05.578 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/291 2024-03-04T12:54:05.578 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/291 2024-03-04T12:54:05.578 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:05.583 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:05.959 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:05.959 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-03-04T12:54:06.139 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:06.142 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:06.144 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:06.485 INFO:teuthology.orchestra.run.smithi148.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:06.491 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/292 2024-03-04T12:54:06.491 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/292 2024-03-04T12:54:06.491 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:06.496 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:06.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:06.870 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-03-04T12:54:07.046 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:07.050 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:07.050 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:07.051 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:07.051 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:07.408 INFO:teuthology.orchestra.run.smithi148.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:07.414 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/293 2024-03-04T12:54:07.414 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/293 2024-03-04T12:54:07.414 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:07.799 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:07.799 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-03-04T12:54:07.979 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:07.982 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:07.983 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:07.983 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:07.984 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:08.477 INFO:teuthology.orchestra.run.smithi148.stdout:generic/294 0s 2024-03-04T12:54:08.486 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/294 2024-03-04T12:54:08.486 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:08.503 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:08.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:08.516 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-03-04T12:54:08.877 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:08.881 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:08.881 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:08.882 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:09.283 INFO:teuthology.orchestra.run.smithi148.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:09.290 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/295 2024-03-04T12:54:09.290 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/295 2024-03-04T12:54:09.290 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:09.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:09.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:09.658 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-03-04T12:54:09.836 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:09.840 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:09.842 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:09.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:10.216 INFO:teuthology.orchestra.run.smithi148.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:10.221 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/296 2024-03-04T12:54:10.221 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/296 2024-03-04T12:54:10.221 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:10.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:10.636 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-03-04T12:54:10.816 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:10.819 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:10.820 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:10.821 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:11.210 INFO:teuthology.orchestra.run.smithi148.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:11.215 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/297 2024-03-04T12:54:11.216 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/297 2024-03-04T12:54:11.216 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:11.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:11.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:11.604 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-03-04T12:54:11.786 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:11.790 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:11.791 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:11.791 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:11.792 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:12.162 INFO:teuthology.orchestra.run.smithi148.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:12.167 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/298 2024-03-04T12:54:12.167 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/298 2024-03-04T12:54:12.167 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:12.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:12.571 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-03-04T12:54:12.745 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:12.749 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:12.749 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:12.750 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:12.750 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:13.097 INFO:teuthology.orchestra.run.smithi148.stdout:generic/299 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:54:13.101 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/299 2024-03-04T12:54:13.101 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/299 2024-03-04T12:54:13.101 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:13.103 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:13.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:13.119 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-03-04T12:54:13.301 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:13.304 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:13.305 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:13.306 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:13.306 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:13.588 INFO:teuthology.orchestra.run.smithi148.stdout:generic/300 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:54:13.592 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/300 2024-03-04T12:54:13.592 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/300 2024-03-04T12:54:13.592 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:13.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:13.614 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-03-04T12:54:13.789 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:13.793 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:13.793 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:13.794 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:14.147 INFO:teuthology.orchestra.run.smithi148.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:14.154 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/301 2024-03-04T12:54:14.154 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/301 2024-03-04T12:54:14.154 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:14.157 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:14.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:14.544 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-03-04T12:54:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:14.730 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:14.731 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:14.732 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:15.078 INFO:teuthology.orchestra.run.smithi148.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:15.085 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/302 2024-03-04T12:54:15.085 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/302 2024-03-04T12:54:15.085 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:15.089 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:15.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:15.471 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-03-04T12:54:15.648 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:15.651 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:15.652 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:15.652 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:15.652 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:15.988 INFO:teuthology.orchestra.run.smithi148.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:54:15.992 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/303 2024-03-04T12:54:15.992 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/303 2024-03-04T12:54:15.992 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:15.994 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:16.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:16.050 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-03-04T12:54:16.521 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:16.524 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:16.525 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:16.526 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:16.526 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:16.765 INFO:teuthology.orchestra.run.smithi148.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:54:16.768 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/304 2024-03-04T12:54:16.769 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/304 2024-03-04T12:54:16.769 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:16.771 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:16.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:16.822 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-03-04T12:54:17.269 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:17.272 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:17.273 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:17.275 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:17.275 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:17.619 INFO:teuthology.orchestra.run.smithi148.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:54:17.626 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/305 2024-03-04T12:54:17.626 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/305 2024-03-04T12:54:17.626 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:17.628 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:18.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:18.030 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-03-04T12:54:18.204 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:18.208 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:18.209 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:18.860 INFO:teuthology.orchestra.run.smithi148.stdout:generic/306 0s 2024-03-04T12:54:18.872 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/306 2024-03-04T12:54:18.873 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:18.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:19.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:19.102 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-03-04T12:54:19.576 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:19.579 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:19.580 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:19.581 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:19.581 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:21.856 INFO:teuthology.orchestra.run.smithi148.stdout:generic/307 2s 2024-03-04T12:54:21.868 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/307 2024-03-04T12:54:21.868 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:21.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:21.879 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-03-04T12:54:22.053 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:22.057 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:22.057 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:22.058 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:22.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:22.488 INFO:teuthology.orchestra.run.smithi148.stdout:generic/308 0s 2024-03-04T12:54:22.496 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/308 2024-03-04T12:54:22.496 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:22.498 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:22.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:22.515 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-03-04T12:54:22.698 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:22.702 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:22.702 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:22.703 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:22.703 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:23.970 INFO:teuthology.orchestra.run.smithi148.stdout:generic/309 1s 2024-03-04T12:54:23.975 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/309 2024-03-04T12:54:23.976 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:54:23.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:54:24.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:54:24.023 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-03-04T12:54:24.505 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:54:24.508 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:54:24.509 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:54:24.510 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:54:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:36.203 INFO:teuthology.orchestra.run.smithi148.stdout:generic/310 72s 2024-03-04T12:55:36.218 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/310 2024-03-04T12:55:36.218 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:36.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:36.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:36.350 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-03-04T12:55:36.796 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:36.799 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:36.800 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:36.801 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:36.801 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:37.091 INFO:teuthology.orchestra.run.smithi148.stdout:generic/311 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:55:37.095 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/311 2024-03-04T12:55:37.095 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/311 2024-03-04T12:55:37.095 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:37.097 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:37.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:37.114 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-03-04T12:55:37.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:55:37.212 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T12:55:37.243 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T12:55:37.237+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 304278) UID: 0 2024-03-04T12:55:37.243 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T12:55:37.237+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 304278) UID: 0 2024-03-04T12:55:37.244 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T12:55:37.237+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 304278) UID: 0 2024-03-04T12:55:37.244 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T12:55:37.237+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 304278) UID: 0 2024-03-04T12:55:37.303 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:37.307 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:37.308 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:37.309 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:37.309 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:37.538 INFO:teuthology.orchestra.run.smithi148.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:55:37.542 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/312 2024-03-04T12:55:37.542 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/312 2024-03-04T12:55:37.542 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:37.544 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:37.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:37.561 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-03-04T12:55:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:37.793 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:37.794 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:37.795 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:37.795 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:42.074 INFO:teuthology.orchestra.run.smithi148.stdout:generic/313 5s 2024-03-04T12:55:42.079 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/313 2024-03-04T12:55:42.079 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:42.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:42.098 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-03-04T12:55:42.273 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:42.277 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:42.277 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:42.278 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:42.278 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:43.262 INFO:teuthology.orchestra.run.smithi148.stdout:generic/314 1s 2024-03-04T12:55:43.267 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/314 2024-03-04T12:55:43.267 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:43.270 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:43.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:43.289 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-03-04T12:55:43.475 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:43.479 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:43.479 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:43.480 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:43.699 INFO:teuthology.orchestra.run.smithi148.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T12:55:43.703 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/315 2024-03-04T12:55:43.704 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/315 2024-03-04T12:55:43.704 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:43.706 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:43.723 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:43.723 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-03-04T12:55:43.905 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:43.909 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:43.911 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:43.911 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:43.911 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:44.255 INFO:teuthology.orchestra.run.smithi148.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T12:55:44.259 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/316 2024-03-04T12:55:44.259 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/316 2024-03-04T12:55:44.259 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:44.261 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:44.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:44.317 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-03-04T12:55:44.707 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:44.712 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:44.712 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:44.713 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:44.713 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:48.293 INFO:teuthology.orchestra.run.smithi148.stdout:generic/317 4s 2024-03-04T12:55:48.299 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/317 2024-03-04T12:55:48.299 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:48.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:48.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:48.310 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-03-04T12:55:48.487 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:48.491 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:48.491 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:48.492 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:48.492 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:49.151 INFO:teuthology.orchestra.run.smithi148.stdout:generic/318 1s 2024-03-04T12:55:49.157 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/318 2024-03-04T12:55:49.157 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:49.159 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:49.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:49.168 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-03-04T12:55:49.348 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:49.351 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:49.353 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:49.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:49.697 INFO:teuthology.orchestra.run.smithi148.stdout:generic/319 0s 2024-03-04T12:55:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/319 2024-03-04T12:55:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:49.704 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:49.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:49.713 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-03-04T12:55:49.897 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:49.901 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:49.902 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:49.902 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:49.903 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.118 INFO:teuthology.orchestra.run.smithi148.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:55:50.122 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/320 2024-03-04T12:55:50.122 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/320 2024-03-04T12:55:50.122 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:50.137 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-03-04T12:55:50.317 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:50.320 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:50.321 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:50.322 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.543 INFO:teuthology.orchestra.run.smithi148.stdout:generic/321 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:55:50.547 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/321 2024-03-04T12:55:50.547 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/321 2024-03-04T12:55:50.547 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:50.562 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-03-04T12:55:50.748 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:50.752 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:50.753 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:50.754 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.977 INFO:teuthology.orchestra.run.smithi148.stdout:generic/322 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:55:50.981 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/322 2024-03-04T12:55:50.981 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/322 2024-03-04T12:55:50.981 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:55:50.983 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:55:50.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:55:50.997 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-03-04T12:55:51.171 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:55:51.175 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:55:51.175 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:55:51.176 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:55:51.176 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:52.527 INFO:teuthology.orchestra.run.smithi148.stdout:generic/323 121s 2024-03-04T12:57:52.539 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/323 2024-03-04T12:57:52.539 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:52.542 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:52.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:52.579 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-03-04T12:57:53.267 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:53.270 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:53.271 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:53.272 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:53.272 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:53.727 INFO:teuthology.orchestra.run.smithi148.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2024-03-04T12:57:53.731 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/324 2024-03-04T12:57:53.731 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/324 2024-03-04T12:57:53.731 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:53.734 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:53.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:53.747 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-03-04T12:57:53.918 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:54.300 INFO:teuthology.orchestra.run.smithi148.stdout:generic/325 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:57:54.304 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/325 2024-03-04T12:57:54.304 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/325 2024-03-04T12:57:54.304 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:54.306 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:54.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:54.320 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-03-04T12:57:54.493 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:54.497 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:54.497 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:54.498 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:54.955 INFO:teuthology.orchestra.run.smithi148.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:54.960 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/326 2024-03-04T12:57:54.960 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/326 2024-03-04T12:57:54.960 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:55.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:55.293 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-03-04T12:57:55.476 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:55.480 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:55.480 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:55.481 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:55.481 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:55.977 INFO:teuthology.orchestra.run.smithi148.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:55.983 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/327 2024-03-04T12:57:55.983 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/327 2024-03-04T12:57:55.983 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:55.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:56.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:56.318 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-03-04T12:57:56.494 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:56.498 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:56.498 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:56.499 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:56.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:56.847 INFO:teuthology.orchestra.run.smithi148.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:56.854 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/328 2024-03-04T12:57:56.854 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/328 2024-03-04T12:57:56.854 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:56.856 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:57.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:57.157 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-03-04T12:57:57.340 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:57.344 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:57.344 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:57.345 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:57.345 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:57.771 INFO:teuthology.orchestra.run.smithi148.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:57.778 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/329 2024-03-04T12:57:57.778 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/329 2024-03-04T12:57:57.778 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:58.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:58.107 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-03-04T12:57:58.284 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:58.288 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:58.288 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:58.289 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:58.722 INFO:teuthology.orchestra.run.smithi148.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:58.730 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/330 2024-03-04T12:57:58.730 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/330 2024-03-04T12:57:58.731 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:58.734 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:59.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:59.082 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-03-04T12:57:59.256 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:57:59.259 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:57:59.260 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:57:59.261 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:57:59.261 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:59.654 INFO:teuthology.orchestra.run.smithi148.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:57:59.659 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/331 2024-03-04T12:57:59.659 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/331 2024-03-04T12:57:59.659 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:57:59.661 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:57:59.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:57:59.995 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-03-04T12:58:00.172 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:00.176 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:00.176 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:00.177 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:00.177 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:00.551 INFO:teuthology.orchestra.run.smithi148.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:58:00.561 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/332 2024-03-04T12:58:00.561 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/332 2024-03-04T12:58:00.561 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:00.565 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:00.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:00.884 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-03-04T12:58:01.074 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:01.078 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:01.078 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:01.079 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:01.079 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:01.454 INFO:teuthology.orchestra.run.smithi148.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:58:01.459 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/333 2024-03-04T12:58:01.460 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/333 2024-03-04T12:58:01.460 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:01.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:01.839 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-03-04T12:58:02.010 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:02.013 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:02.014 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:02.015 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:02.015 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:02.363 INFO:teuthology.orchestra.run.smithi148.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:58:02.370 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/334 2024-03-04T12:58:02.370 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/334 2024-03-04T12:58:02.370 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:02.372 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:02.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:02.763 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-03-04T12:58:02.948 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:02.952 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:02.953 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:02.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:03.177 INFO:teuthology.orchestra.run.smithi148.stdout:generic/335 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:03.181 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/335 2024-03-04T12:58:03.181 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/335 2024-03-04T12:58:03.181 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:03.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:03.196 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-03-04T12:58:03.379 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:03.383 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:03.383 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:03.384 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:03.384 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:03.626 INFO:teuthology.orchestra.run.smithi148.stdout:generic/336 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:03.630 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/336 2024-03-04T12:58:03.630 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/336 2024-03-04T12:58:03.630 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:03.632 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:03.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:03.650 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-03-04T12:58:03.835 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:03.839 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:03.839 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:03.840 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:03.840 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:04.207 INFO:teuthology.orchestra.run.smithi148.stdout:generic/337 1s 2024-03-04T12:58:04.211 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/337 2024-03-04T12:58:04.212 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:04.214 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:04.222 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:04.222 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-03-04T12:58:04.409 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:04.413 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:04.413 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:04.414 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:04.635 INFO:teuthology.orchestra.run.smithi148.stdout:generic/338 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:04.639 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/338 2024-03-04T12:58:04.639 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/338 2024-03-04T12:58:04.639 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:04.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:04.655 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-03-04T12:58:04.836 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:04.839 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:04.840 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:04.841 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:53.232 INFO:teuthology.orchestra.run.smithi148.stdout:generic/339 49s 2024-03-04T12:58:53.237 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/339 2024-03-04T12:58:53.237 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:53.239 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:53.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:53.248 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-03-04T12:58:53.418 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:53.422 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:53.422 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:53.423 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:55.918 INFO:teuthology.orchestra.run.smithi148.stdout:generic/340 2s 2024-03-04T12:58:55.925 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/340 2024-03-04T12:58:55.925 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:55.931 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:55.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:55.947 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-03-04T12:58:56.339 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:56.343 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:56.343 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:56.344 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:56.344 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:56.694 INFO:teuthology.orchestra.run.smithi148.stdout:generic/341 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:56.698 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/341 2024-03-04T12:58:56.698 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/341 2024-03-04T12:58:56.698 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:56.700 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:56.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:56.714 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-03-04T12:58:56.922 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:56.927 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:56.927 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:56.929 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:56.929 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:57.224 INFO:teuthology.orchestra.run.smithi148.stdout:generic/342 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:57.228 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/342 2024-03-04T12:58:57.228 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/342 2024-03-04T12:58:57.228 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:57.233 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:57.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:57.259 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-03-04T12:58:57.483 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:57.487 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:57.488 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:57.489 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:57.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:57.742 INFO:teuthology.orchestra.run.smithi148.stdout:generic/343 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:58:57.746 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/343 2024-03-04T12:58:57.746 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/343 2024-03-04T12:58:57.746 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:58:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:58:57.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:58:57.764 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-03-04T12:58:57.942 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:58:57.946 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:58:57.946 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:58:57.947 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:58:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:02.758 INFO:teuthology.orchestra.run.smithi148.stdout:generic/344 4s 2024-03-04T12:59:02.768 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/344 2024-03-04T12:59:02.768 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:02.770 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:02.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:02.786 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-03-04T12:59:03.215 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:03.220 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:03.220 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:03.222 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:03.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:08.548 INFO:teuthology.orchestra.run.smithi148.stdout:generic/345 5s 2024-03-04T12:59:08.557 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/345 2024-03-04T12:59:08.557 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:08.562 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:08.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:08.579 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-03-04T12:59:08.994 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:09.001 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:09.002 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:09.005 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:09.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:11.663 INFO:teuthology.orchestra.run.smithi148.stdout:generic/346 2s 2024-03-04T12:59:11.671 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/346 2024-03-04T12:59:11.671 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:11.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:11.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:11.695 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-03-04T12:59:12.069 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:12.076 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:12.076 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:12.077 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:12.428 INFO:teuthology.orchestra.run.smithi148.stdout:generic/347 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:59:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/347 2024-03-04T12:59:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/347 2024-03-04T12:59:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:12.439 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:12.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:12.458 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-03-04T12:59:12.676 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:12.680 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:12.680 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:12.681 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:12.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:13.003 INFO:teuthology.orchestra.run.smithi148.stdout:generic/348 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:59:13.007 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/348 2024-03-04T12:59:13.007 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/348 2024-03-04T12:59:13.007 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:13.009 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:13.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:13.025 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-03-04T12:59:13.197 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:13.200 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:13.201 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:13.202 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:13.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:13.720 INFO:teuthology.orchestra.run.smithi148.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-03-04T12:59:13.724 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/349 2024-03-04T12:59:13.724 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/349 2024-03-04T12:59:13.724 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:13.725 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:13.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:13.742 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-03-04T12:59:13.935 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:13.939 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:13.939 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:13.940 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:13.940 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:14.649 INFO:teuthology.orchestra.run.smithi148.stdout:generic/350 0s 2024-03-04T12:59:14.654 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/350 2024-03-04T12:59:14.654 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:14.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:14.664 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-03-04T12:59:14.889 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:14.893 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:14.894 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:14.895 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:15.126 INFO:teuthology.orchestra.run.smithi148.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:59:15.130 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/351 2024-03-04T12:59:15.131 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/351 2024-03-04T12:59:15.131 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:15.133 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:15.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:15.147 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-03-04T12:59:15.348 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:15.352 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:15.352 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:15.353 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:15.728 INFO:teuthology.orchestra.run.smithi148.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:15.735 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/352 2024-03-04T12:59:15.735 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/352 2024-03-04T12:59:15.736 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:15.737 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:16.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:16.095 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-03-04T12:59:16.269 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:16.273 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:16.273 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:16.274 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:16.274 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:16.634 INFO:teuthology.orchestra.run.smithi148.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:16.640 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/353 2024-03-04T12:59:16.640 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/353 2024-03-04T12:59:16.640 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:16.643 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:16.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:16.994 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-03-04T12:59:17.173 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:17.177 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:17.177 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:17.178 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:17.178 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:21.612 INFO:teuthology.orchestra.run.smithi148.stdout:generic/354 4s 2024-03-04T12:59:21.643 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/354 2024-03-04T12:59:21.643 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:21.645 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:21.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:21.654 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-03-04T12:59:21.840 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:21.844 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:21.844 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:21.845 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:23.004 INFO:teuthology.orchestra.run.smithi148.stdout:generic/355 1s 2024-03-04T12:59:23.010 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/355 2024-03-04T12:59:23.010 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:23.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:23.057 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-03-04T12:59:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:23.476 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:23.876 INFO:teuthology.orchestra.run.smithi148.stdout:generic/356 [not run] swapfiles are not supported 2024-03-04T12:59:23.880 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/356 2024-03-04T12:59:23.880 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/356 2024-03-04T12:59:23.880 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:23.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:23.896 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-03-04T12:59:24.080 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:24.083 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:24.084 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:24.085 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:24.414 INFO:teuthology.orchestra.run.smithi148.stdout:generic/357 [not run] swapfiles are not supported 2024-03-04T12:59:24.418 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/357 2024-03-04T12:59:24.418 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/357 2024-03-04T12:59:24.418 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:24.420 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:24.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:24.433 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-03-04T12:59:24.610 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:24.614 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:24.614 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:24.615 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:24.615 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:24.973 INFO:teuthology.orchestra.run.smithi148.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:24.980 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/358 2024-03-04T12:59:24.980 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/358 2024-03-04T12:59:24.980 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:25.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:25.307 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-03-04T12:59:25.524 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:25.528 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:25.528 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:25.529 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:25.529 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:25.891 INFO:teuthology.orchestra.run.smithi148.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:25.898 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/359 2024-03-04T12:59:25.898 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/359 2024-03-04T12:59:25.898 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:26.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:26.217 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-03-04T12:59:26.407 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:26.411 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:26.412 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:26.413 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:26.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:26.668 INFO:teuthology.orchestra.run.smithi148.stdout:generic/360 0s 2024-03-04T12:59:26.673 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/360 2024-03-04T12:59:26.673 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:26.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:26.693 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-03-04T12:59:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:26.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.112 INFO:teuthology.orchestra.run.smithi148.stdout:generic/361 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:59:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/361 2024-03-04T12:59:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/361 2024-03-04T12:59:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:27.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.130 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:27.130 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-03-04T12:59:27.307 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:27.311 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:27.311 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:27.312 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.515 INFO:teuthology.orchestra.run.smithi148.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:27.519 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/362 2024-03-04T12:59:27.519 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/362 2024-03-04T12:59:27.519 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:27.521 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:27.534 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-03-04T12:59:27.712 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:27.716 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:27.716 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:27.717 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.916 INFO:teuthology.orchestra.run.smithi148.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:27.920 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/363 2024-03-04T12:59:27.920 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/363 2024-03-04T12:59:27.920 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:27.923 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:27.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:27.935 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-03-04T12:59:28.118 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:28.122 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:28.122 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:28.123 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:28.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:28.372 INFO:teuthology.orchestra.run.smithi148.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:28.376 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/364 2024-03-04T12:59:28.376 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/364 2024-03-04T12:59:28.376 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:28.378 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:28.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:28.401 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-03-04T12:59:28.584 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:28.588 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:28.589 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:28.589 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:28.791 INFO:teuthology.orchestra.run.smithi148.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:28.795 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/365 2024-03-04T12:59:28.795 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/365 2024-03-04T12:59:28.795 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:28.820 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:28.821 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-03-04T12:59:29.004 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:29.008 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:29.008 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:29.009 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:29.009 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:29.206 INFO:teuthology.orchestra.run.smithi148.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:29.210 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/366 2024-03-04T12:59:29.210 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/366 2024-03-04T12:59:29.210 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:29.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:29.225 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-03-04T12:59:29.406 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:29.410 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:29.410 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:29.411 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:29.411 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:29.612 INFO:teuthology.orchestra.run.smithi148.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:29.616 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/367 2024-03-04T12:59:29.616 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/367 2024-03-04T12:59:29.616 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:29.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:29.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:29.630 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-03-04T12:59:29.810 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:29.813 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:29.814 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:29.815 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:29.815 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.026 INFO:teuthology.orchestra.run.smithi148.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:30.030 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/368 2024-03-04T12:59:30.030 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/368 2024-03-04T12:59:30.030 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:30.032 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.044 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:30.045 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-03-04T12:59:30.217 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:30.221 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:30.221 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:30.222 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:30.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.424 INFO:teuthology.orchestra.run.smithi148.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:30.428 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/369 2024-03-04T12:59:30.428 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/369 2024-03-04T12:59:30.428 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:30.430 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:30.442 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-03-04T12:59:30.617 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:30.620 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:30.621 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:30.622 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:30.622 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.820 INFO:teuthology.orchestra.run.smithi148.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2024-03-04T12:59:30.823 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/370 2024-03-04T12:59:30.823 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/370 2024-03-04T12:59:30.824 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:30.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:30.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:30.851 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-03-04T12:59:31.024 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:31.028 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:31.028 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:31.029 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:31.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:31.246 INFO:teuthology.orchestra.run.smithi148.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:59:31.250 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/371 2024-03-04T12:59:31.250 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/371 2024-03-04T12:59:31.250 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:31.252 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:31.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:31.267 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-03-04T12:59:31.442 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:31.445 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:31.446 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:31.446 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:31.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:31.799 INFO:teuthology.orchestra.run.smithi148.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:31.804 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/372 2024-03-04T12:59:31.804 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/372 2024-03-04T12:59:31.804 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:31.807 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:32.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:32.151 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-03-04T12:59:32.323 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:32.325 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:32.325 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:32.326 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:32.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:32.677 INFO:teuthology.orchestra.run.smithi148.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:32.684 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/373 2024-03-04T12:59:32.684 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/373 2024-03-04T12:59:32.684 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:32.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:33.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:33.009 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-03-04T12:59:33.197 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:33.200 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:33.201 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:33.202 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:33.511 INFO:teuthology.orchestra.run.smithi148.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-03-04T12:59:33.556 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/374 2024-03-04T12:59:33.556 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/374 2024-03-04T12:59:33.556 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:33.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:33.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:33.940 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-03-04T12:59:34.130 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:34.134 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:34.134 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:34.135 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:34.456 INFO:teuthology.orchestra.run.smithi148.stdout:generic/375 0s 2024-03-04T12:59:34.461 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/375 2024-03-04T12:59:34.461 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:34.463 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:34.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:34.521 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-03-04T12:59:34.920 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:34.924 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:34.924 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:34.925 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:35.143 INFO:teuthology.orchestra.run.smithi148.stdout:generic/376 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:59:35.147 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/376 2024-03-04T12:59:35.147 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/376 2024-03-04T12:59:35.147 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:35.149 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:35.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:35.161 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-03-04T12:59:35.348 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:35.351 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:35.352 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:35.353 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:35.737 INFO:teuthology.orchestra.run.smithi148.stdout:generic/377 0s 2024-03-04T12:59:35.742 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/377 2024-03-04T12:59:35.742 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:35.744 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:35.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:35.753 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-03-04T12:59:35.931 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:35.935 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:35.936 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:35.936 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:36.386 INFO:teuthology.orchestra.run.smithi148.stdout:generic/378 0s 2024-03-04T12:59:36.391 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/378 2024-03-04T12:59:36.391 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:36.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:36.446 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-03-04T12:59:36.824 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:36.829 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:36.830 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:36.831 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:36.831 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.070 INFO:teuthology.orchestra.run.smithi148.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:37.074 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/379 2024-03-04T12:59:37.074 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/379 2024-03-04T12:59:37.074 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:37.076 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:37.099 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-03-04T12:59:37.283 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:37.286 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:37.287 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:37.288 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:37.288 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.493 INFO:teuthology.orchestra.run.smithi148.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:37.497 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/380 2024-03-04T12:59:37.497 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/380 2024-03-04T12:59:37.497 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:37.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:37.511 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-03-04T12:59:37.691 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:37.695 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:37.695 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:37.696 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.893 INFO:teuthology.orchestra.run.smithi148.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:37.897 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/381 2024-03-04T12:59:37.897 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/381 2024-03-04T12:59:37.897 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:37.899 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:37.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:37.911 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-03-04T12:59:38.097 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:38.100 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:38.101 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:38.102 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:38.102 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:38.307 INFO:teuthology.orchestra.run.smithi148.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:38.311 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/382 2024-03-04T12:59:38.311 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/382 2024-03-04T12:59:38.311 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:38.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:38.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:38.325 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-03-04T12:59:38.506 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:38.510 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:38.510 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:38.511 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:38.511 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:38.716 INFO:teuthology.orchestra.run.smithi148.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:38.720 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/383 2024-03-04T12:59:38.720 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/383 2024-03-04T12:59:38.720 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:38.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:38.735 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-03-04T12:59:38.919 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:38.922 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:38.923 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:38.924 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.127 INFO:teuthology.orchestra.run.smithi148.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:39.131 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/384 2024-03-04T12:59:39.131 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/384 2024-03-04T12:59:39.131 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:39.133 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:39.145 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-03-04T12:59:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:39.325 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:39.326 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:39.327 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.530 INFO:teuthology.orchestra.run.smithi148.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:39.534 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/385 2024-03-04T12:59:39.534 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/385 2024-03-04T12:59:39.534 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.549 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:39.549 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-03-04T12:59:39.733 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:39.736 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:39.737 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:39.738 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:39.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.938 INFO:teuthology.orchestra.run.smithi148.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:39.942 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/386 2024-03-04T12:59:39.942 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/386 2024-03-04T12:59:39.942 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:39.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:39.956 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-03-04T12:59:40.138 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:40.142 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:40.143 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:40.144 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:40.484 INFO:teuthology.orchestra.run.smithi148.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:40.491 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/387 2024-03-04T12:59:40.491 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/387 2024-03-04T12:59:40.491 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:40.493 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:40.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:40.862 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-03-04T12:59:41.042 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:41.045 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:41.046 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:41.047 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:41.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:41.272 INFO:teuthology.orchestra.run.smithi148.stdout:generic/388 [not run] require 172.21.15.148:6789:/scratch to be local device 2024-03-04T12:59:41.276 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/388 2024-03-04T12:59:41.276 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/388 2024-03-04T12:59:41.276 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:41.278 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:41.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:41.291 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-03-04T12:59:41.466 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:41.469 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:41.470 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:41.471 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:41.471 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:41.678 INFO:teuthology.orchestra.run.smithi148.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-03-04T12:59:41.681 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/389 2024-03-04T12:59:41.682 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/389 2024-03-04T12:59:41.682 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:41.695 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:41.696 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-03-04T12:59:41.883 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:41.886 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:41.886 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:41.887 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:42.128 INFO:teuthology.orchestra.run.smithi148.stdout:generic/390 [not run] ceph does not support freezing 2024-03-04T12:59:42.132 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/390 2024-03-04T12:59:42.132 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/390 2024-03-04T12:59:42.132 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:42.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:42.155 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-03-04T12:59:42.333 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:42.340 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:42.340 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:42.340 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:42.340 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:42.555 INFO:teuthology.orchestra.run.smithi148.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:59:42.559 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/391 2024-03-04T12:59:42.559 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/391 2024-03-04T12:59:42.559 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:42.561 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:42.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:42.592 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-03-04T12:59:42.771 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:42.775 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:42.775 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:42.776 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:42.776 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:43.094 INFO:teuthology.orchestra.run.smithi148.stdout:generic/392 [not run] ceph does not support shutdown 2024-03-04T12:59:43.113 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/392 2024-03-04T12:59:43.113 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/392 2024-03-04T12:59:43.113 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:43.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:43.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:43.127 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-03-04T12:59:43.302 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:43.306 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:43.306 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:43.307 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:43.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:43.919 INFO:teuthology.orchestra.run.smithi148.stdout:generic/393 0s 2024-03-04T12:59:43.924 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/393 2024-03-04T12:59:43.924 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:43.926 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:43.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:43.935 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-03-04T12:59:44.109 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:44.113 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:44.113 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:44.114 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:44.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:44.352 INFO:teuthology.orchestra.run.smithi148.stdout:generic/394 0s 2024-03-04T12:59:44.357 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/394 2024-03-04T12:59:44.357 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:44.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:44.409 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-03-04T12:59:44.778 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:44.784 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:44.784 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:44.785 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:44.786 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:45.218 INFO:teuthology.orchestra.run.smithi148.stdout:generic/395 [not run] kernel does not support ceph encryption 2024-03-04T12:59:45.238 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/395 2024-03-04T12:59:45.238 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/395 2024-03-04T12:59:45.238 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:45.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:45.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:45.258 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-03-04T12:59:45.533 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:45.539 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:45.539 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:45.541 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:45.967 INFO:teuthology.orchestra.run.smithi148.stdout:generic/396 [not run] kernel does not support ceph encryption 2024-03-04T12:59:45.985 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/396 2024-03-04T12:59:45.985 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/396 2024-03-04T12:59:45.985 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:46.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:46.005 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-03-04T12:59:46.253 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:46.257 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:46.257 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:46.258 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:46.605 INFO:teuthology.orchestra.run.smithi148.stdout:generic/397 [not run] kernel does not support ceph encryption 2024-03-04T12:59:46.612 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/397 2024-03-04T12:59:46.612 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/397 2024-03-04T12:59:46.612 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:46.614 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:46.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:46.627 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-03-04T12:59:46.805 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:46.809 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:46.809 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:46.810 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:47.213 INFO:teuthology.orchestra.run.smithi148.stdout:generic/398 [not run] kernel does not support ceph encryption 2024-03-04T12:59:47.221 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/398 2024-03-04T12:59:47.221 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/398 2024-03-04T12:59:47.221 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:47.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:47.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:47.235 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-03-04T12:59:47.411 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:47.414 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:47.415 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:47.416 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:47.734 INFO:teuthology.orchestra.run.smithi148.stdout:generic/399 [not run] kernel does not support ceph encryption 2024-03-04T12:59:47.752 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/399 2024-03-04T12:59:47.752 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/399 2024-03-04T12:59:47.752 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:47.754 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:47.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:47.766 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-03-04T12:59:47.948 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:47.951 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:47.952 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:47.953 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:47.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:48.155 INFO:teuthology.orchestra.run.smithi148.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T12:59:48.158 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/400 2024-03-04T12:59:48.158 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/400 2024-03-04T12:59:48.158 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:48.161 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:48.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:48.186 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-03-04T12:59:48.364 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:48.368 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:48.368 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:48.369 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:48.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:48.895 INFO:teuthology.orchestra.run.smithi148.stdout:generic/401 0s 2024-03-04T12:59:48.915 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/401 2024-03-04T12:59:48.915 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:48.919 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:48.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:48.929 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-03-04T12:59:49.262 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:49.266 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:49.266 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:49.267 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:49.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:49.602 INFO:teuthology.orchestra.run.smithi148.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2024-03-04T12:59:49.609 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/402 2024-03-04T12:59:49.609 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/402 2024-03-04T12:59:49.609 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:49.611 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:49.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:49.623 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-03-04T12:59:49.800 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:49.804 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:49.804 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:49.805 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:50.290 INFO:teuthology.orchestra.run.smithi148.stdout:generic/403 1s 2024-03-04T12:59:50.296 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/403 2024-03-04T12:59:50.296 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:50.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:50.307 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-03-04T12:59:50.482 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:50.486 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:50.486 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:50.487 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:50.488 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:50.705 INFO:teuthology.orchestra.run.smithi148.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T12:59:50.711 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/404 2024-03-04T12:59:50.711 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/404 2024-03-04T12:59:50.711 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:50.712 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:50.726 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:50.727 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-03-04T12:59:50.903 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:50.906 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:50.907 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:50.908 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:50.908 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:51.200 INFO:teuthology.orchestra.run.smithi148.stdout:generic/405 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T12:59:51.204 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/405 2024-03-04T12:59:51.204 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/405 2024-03-04T12:59:51.204 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:51.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:51.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:51.219 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-03-04T12:59:51.394 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:51.398 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:51.398 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:51.399 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:53.014 INFO:teuthology.orchestra.run.smithi148.stdout:generic/406 1s 2024-03-04T12:59:53.022 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/406 2024-03-04T12:59:53.022 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:53.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:53.033 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-03-04T12:59:53.213 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:53.217 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:53.217 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:53.219 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:53.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:53.470 INFO:teuthology.orchestra.run.smithi148.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T12:59:53.473 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/407 2024-03-04T12:59:53.474 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/407 2024-03-04T12:59:53.474 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:53.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:53.526 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-03-04T12:59:53.960 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:53.967 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:53.967 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:53.968 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:54.271 INFO:teuthology.orchestra.run.smithi148.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T12:59:54.275 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/408 2024-03-04T12:59:54.275 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/408 2024-03-04T12:59:54.275 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:54.277 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:54.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:54.324 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-03-04T12:59:54.745 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:54.749 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:54.749 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:54.750 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:54.750 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:55.117 INFO:teuthology.orchestra.run.smithi148.stdout:generic/409 [not run] require 172.21.15.148:6789:/scratch to be local device 2024-03-04T12:59:55.121 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/409 2024-03-04T12:59:55.121 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/409 2024-03-04T12:59:55.121 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:55.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:55.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:55.151 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-03-04T12:59:55.334 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:55.338 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:55.339 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:55.339 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:55.340 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:55.575 INFO:teuthology.orchestra.run.smithi148.stdout:generic/410 [not run] require 172.21.15.148:6789:/scratch to be local device 2024-03-04T12:59:55.578 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/410 2024-03-04T12:59:55.578 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/410 2024-03-04T12:59:55.579 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:55.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:55.593 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-03-04T12:59:55.765 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:55.769 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:55.769 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:55.770 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:55.770 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:56.168 INFO:teuthology.orchestra.run.smithi148.stdout:generic/411 [not run] require 172.21.15.148:6789:/scratch to be local device 2024-03-04T12:59:56.171 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/411 2024-03-04T12:59:56.171 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/411 2024-03-04T12:59:56.172 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:56.174 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:56.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:56.195 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-03-04T12:59:56.365 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:56.368 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:56.368 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:56.370 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:56.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:56.892 INFO:teuthology.orchestra.run.smithi148.stdout:generic/412 0s 2024-03-04T12:59:56.897 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/412 2024-03-04T12:59:56.897 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:56.899 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:56.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:56.908 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-03-04T12:59:57.094 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:57.098 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:57.098 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:57.382 INFO:teuthology.orchestra.run.smithi148.stdout:generic/413 [not run] mount 172.21.15.148:6789:/scratch with dax failed 2024-03-04T12:59:57.386 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/413 2024-03-04T12:59:57.387 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/413 2024-03-04T12:59:57.387 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:57.389 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:57.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:57.401 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-03-04T12:59:57.582 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:57.585 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:57.586 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:57.587 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:57.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:57.937 INFO:teuthology.orchestra.run.smithi148.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:57.945 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/414 2024-03-04T12:59:57.945 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/414 2024-03-04T12:59:57.945 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:57.950 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:58.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:58.313 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-03-04T12:59:58.483 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:58.487 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:58.488 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:58.489 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:58.844 INFO:teuthology.orchestra.run.smithi148.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T12:59:58.853 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/415 2024-03-04T12:59:58.853 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/415 2024-03-04T12:59:58.853 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:58.860 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:59.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:59.209 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-03-04T12:59:59.394 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:59.397 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:59.398 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:59.399 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:59.399 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:59.656 INFO:teuthology.orchestra.run.smithi148.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T12:59:59.659 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/416 2024-03-04T12:59:59.660 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/416 2024-03-04T12:59:59.660 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T12:59:59.662 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T12:59:59.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T12:59:59.684 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-03-04T12:59:59.865 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T12:59:59.869 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T12:59:59.869 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T12:59:59.870 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T12:59:59.870 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:00.174 INFO:teuthology.orchestra.run.smithi148.stdout:generic/417 [not run] ceph does not support shutdown 2024-03-04T13:00:00.182 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/417 2024-03-04T13:00:00.182 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/417 2024-03-04T13:00:00.182 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:00.184 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:00.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:00.197 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-03-04T13:00:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:00.382 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:00.384 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:00.633 INFO:teuthology.orchestra.run.smithi148.stdout:generic/418 [not run] require 172.21.15.148:6789:/test to be valid block disk 2024-03-04T13:00:00.637 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/418 2024-03-04T13:00:00.637 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/418 2024-03-04T13:00:00.637 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:00.639 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:00.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:00.666 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-03-04T13:00:00.840 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:00.843 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:00.844 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:00.845 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:00.845 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:01.199 INFO:teuthology.orchestra.run.smithi148.stdout:generic/419 [not run] kernel does not support ceph encryption 2024-03-04T13:00:01.217 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/419 2024-03-04T13:00:01.217 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/419 2024-03-04T13:00:01.217 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:01.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:01.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:01.232 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-03-04T13:00:01.412 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:01.416 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:01.416 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:01.417 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:01.763 INFO:teuthology.orchestra.run.smithi148.stdout:generic/420 0s 2024-03-04T13:00:01.768 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/420 2024-03-04T13:00:01.768 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:01.770 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:01.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:01.822 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-03-04T13:00:02.269 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:02.274 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:02.274 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:02.275 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:02.275 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:02.598 INFO:teuthology.orchestra.run.smithi148.stdout:generic/421 [not run] kernel does not support ceph encryption 2024-03-04T13:00:02.605 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/421 2024-03-04T13:00:02.605 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/421 2024-03-04T13:00:02.605 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:02.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:02.619 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-03-04T13:00:02.803 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:02.807 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:02.807 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:02.808 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:02.808 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:03.047 INFO:teuthology.orchestra.run.smithi148.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T13:00:03.052 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/422 2024-03-04T13:00:03.052 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/422 2024-03-04T13:00:03.052 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:03.055 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:03.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:03.085 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-03-04T13:00:03.298 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:03.302 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:03.303 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:03.303 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:03.304 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:03.702 INFO:teuthology.orchestra.run.smithi148.stdout:generic/423 0s 2024-03-04T13:00:03.709 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/423 2024-03-04T13:00:03.709 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:03.711 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:03.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:03.765 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-03-04T13:00:04.203 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:04.207 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:04.208 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:04.209 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:04.210 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:04.467 INFO:teuthology.orchestra.run.smithi148.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-03-04T13:00:04.471 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/424 2024-03-04T13:00:04.471 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/424 2024-03-04T13:00:04.471 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:04.473 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:04.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:04.489 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-03-04T13:00:04.667 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:04.671 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:04.671 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:04.672 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:04.941 INFO:teuthology.orchestra.run.smithi148.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2024-03-04T13:00:04.945 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/425 2024-03-04T13:00:04.945 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/425 2024-03-04T13:00:04.945 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:04.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:04.989 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:04.989 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-03-04T13:00:05.418 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:05.421 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:05.422 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:05.423 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:05.423 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:34.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:00:34.393 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:00:36.837 INFO:teuthology.orchestra.run.smithi148.stdout:generic/426 31s 2024-03-04T13:00:36.991 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/426 2024-03-04T13:00:36.991 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:37.062 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T13:00:37.056+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 358848) UID: 0 2024-03-04T13:00:37.086 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T13:00:37.055+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 358848) UID: 0 2024-03-04T13:00:37.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:37.109 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-03-04T13:00:38.730 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:38.734 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:38.734 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:38.735 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:39.217 INFO:teuthology.orchestra.run.smithi148.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T13:00:39.220 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/427 2024-03-04T13:00:39.220 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/427 2024-03-04T13:00:39.220 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:39.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:39.238 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-03-04T13:00:39.413 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:39.417 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:39.417 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:39.418 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:39.418 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:39.658 INFO:teuthology.orchestra.run.smithi148.stdout:generic/428 0s 2024-03-04T13:00:39.664 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/428 2024-03-04T13:00:39.664 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:39.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:39.685 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-03-04T13:00:39.864 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:39.868 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:39.868 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:39.869 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:39.869 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:40.189 INFO:teuthology.orchestra.run.smithi148.stdout:generic/429 [not run] kernel does not support ceph encryption 2024-03-04T13:00:40.197 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/429 2024-03-04T13:00:40.197 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/429 2024-03-04T13:00:40.197 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:40.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:40.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:40.212 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-03-04T13:00:40.384 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:40.388 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:40.388 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:40.702 INFO:teuthology.orchestra.run.smithi148.stdout:generic/430 0s 2024-03-04T13:00:40.708 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/430 2024-03-04T13:00:40.708 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:40.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:40.774 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-03-04T13:00:41.170 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:41.173 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:41.174 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:41.175 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:41.457 INFO:teuthology.orchestra.run.smithi148.stdout:generic/431 0s 2024-03-04T13:00:41.463 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/431 2024-03-04T13:00:41.463 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:41.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:41.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:41.547 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-03-04T13:00:41.923 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:41.927 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:41.928 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:41.929 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:42.225 INFO:teuthology.orchestra.run.smithi148.stdout:generic/432 0s 2024-03-04T13:00:42.232 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/432 2024-03-04T13:00:42.232 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:42.234 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:42.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:42.293 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-03-04T13:00:42.677 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:42.681 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:42.682 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:42.683 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:42.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:42.950 INFO:teuthology.orchestra.run.smithi148.stdout:generic/433 0s 2024-03-04T13:00:42.955 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/433 2024-03-04T13:00:42.956 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:43.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:43.023 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-03-04T13:00:43.413 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:43.418 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:43.418 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:43.419 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:43.710 INFO:teuthology.orchestra.run.smithi148.stdout:generic/434 0s 2024-03-04T13:00:43.715 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/434 2024-03-04T13:00:43.715 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:43.718 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:43.779 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:43.779 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-03-04T13:00:44.180 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:44.184 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:44.184 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:44.185 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:44.185 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:44.505 INFO:teuthology.orchestra.run.smithi148.stdout:generic/435 [not run] kernel does not support ceph encryption 2024-03-04T13:00:44.527 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/435 2024-03-04T13:00:44.527 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/435 2024-03-04T13:00:44.527 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:44.529 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:44.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:44.542 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-03-04T13:00:44.722 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:44.726 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:44.726 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:44.727 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:44.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:45.236 INFO:teuthology.orchestra.run.smithi148.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T13:00:45.240 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/436 2024-03-04T13:00:45.240 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/436 2024-03-04T13:00:45.240 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:45.242 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:45.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:45.270 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-03-04T13:00:45.638 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:45.644 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:45.645 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:45.646 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:46.488 INFO:teuthology.orchestra.run.smithi148.stdout:generic/437 1s 2024-03-04T13:00:46.494 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/437 2024-03-04T13:00:46.494 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:00:46.496 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:00:46.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:00:46.515 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-03-04T13:00:46.711 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:00:46.714 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:00:46.715 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:00:46.716 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:00:46.716 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:05.366 INFO:teuthology.orchestra.run.smithi148.stdout:generic/438 79s 2024-03-04T13:02:05.370 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/438 2024-03-04T13:02:05.370 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:05.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:05.390 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-03-04T13:02:05.569 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:05.573 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:05.573 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:05.574 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:05.574 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:06.456 INFO:teuthology.orchestra.run.smithi148.stdout:generic/439 1s 2024-03-04T13:02:06.462 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/439 2024-03-04T13:02:06.462 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:06.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:06.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:06.477 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-03-04T13:02:06.659 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:06.663 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:06.663 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:06.664 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:07.000 INFO:teuthology.orchestra.run.smithi148.stdout:generic/440 [not run] kernel does not support ceph encryption 2024-03-04T13:02:07.008 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/440 2024-03-04T13:02:07.008 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/440 2024-03-04T13:02:07.009 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:07.011 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:07.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:07.023 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-03-04T13:02:07.210 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:07.214 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:07.214 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:07.215 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:07.563 INFO:teuthology.orchestra.run.smithi148.stdout:generic/441 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T13:02:07.566 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/441 2024-03-04T13:02:07.567 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/441 2024-03-04T13:02:07.567 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:07.569 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:07.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:07.584 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-03-04T13:02:07.762 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:07.765 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:07.766 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:07.767 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:08.017 INFO:teuthology.orchestra.run.smithi148.stdout:generic/442 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T13:02:08.021 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/442 2024-03-04T13:02:08.021 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/442 2024-03-04T13:02:08.021 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:08.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:08.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:08.037 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-03-04T13:02:08.219 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:08.222 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:08.223 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:08.224 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:08.224 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:08.477 INFO:teuthology.orchestra.run.smithi148.stdout:generic/443 0s 2024-03-04T13:02:08.482 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/443 2024-03-04T13:02:08.482 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:08.485 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:08.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:08.504 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-03-04T13:02:08.684 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:08.688 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:08.688 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:08.689 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:08.689 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:09.076 INFO:teuthology.orchestra.run.smithi148.stdout:generic/444 1s 2024-03-04T13:02:09.082 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/444 2024-03-04T13:02:09.082 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:09.084 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:09.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:09.105 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-03-04T13:02:09.277 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:09.281 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:09.281 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:09.282 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:09.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:09.819 INFO:teuthology.orchestra.run.smithi148.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T13:02:09.823 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/445 2024-03-04T13:02:09.823 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/445 2024-03-04T13:02:09.823 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:09.825 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:09.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:09.863 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-03-04T13:02:10.238 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:10.242 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:10.242 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:10.243 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:10.243 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:27.957 INFO:teuthology.orchestra.run.smithi148.stdout:generic/446 17s 2024-03-04T13:02:27.975 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/446 2024-03-04T13:02:27.975 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:27.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:27.993 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:27.993 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-03-04T13:02:28.333 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:28.337 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:28.337 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:28.338 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:28.338 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:28.713 INFO:teuthology.orchestra.run.smithi148.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T13:02:28.718 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/447 2024-03-04T13:02:28.719 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/447 2024-03-04T13:02:28.719 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:28.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:28.966 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-03-04T13:02:29.213 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:29.217 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:29.217 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:29.218 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:30.173 INFO:teuthology.orchestra.run.smithi148.stdout:generic/448 1s 2024-03-04T13:02:30.179 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/448 2024-03-04T13:02:30.179 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:30.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:30.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:30.235 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-03-04T13:02:30.618 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:30.622 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:30.622 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:30.623 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:30.623 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:30.903 INFO:teuthology.orchestra.run.smithi148.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T13:02:30.906 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/449 2024-03-04T13:02:30.907 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/449 2024-03-04T13:02:30.907 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:30.911 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:30.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:30.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-03-04T13:02:31.393 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:31.397 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:31.397 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:31.398 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:31.398 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:31.973 INFO:teuthology.orchestra.run.smithi148.stdout:generic/450 0s 2024-03-04T13:02:31.979 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/450 2024-03-04T13:02:31.979 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:02:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:02:32.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:02:32.030 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-03-04T13:02:32.408 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:02:32.412 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:02:32.413 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:02:32.414 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:02:32.414 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:02.740 INFO:teuthology.orchestra.run.smithi148.stdout:generic/451 30s 2024-03-04T13:03:02.745 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/451 2024-03-04T13:03:02.746 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:02.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:02.797 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:02.797 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-03-04T13:03:03.194 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:03.198 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:03.198 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:03.199 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:03.581 INFO:teuthology.orchestra.run.smithi148.stdout:generic/452 0s 2024-03-04T13:03:03.586 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/452 2024-03-04T13:03:03.587 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:03.589 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:03.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:03.597 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-03-04T13:03:03.775 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:03.779 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:03.779 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:03.779 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:03.780 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:03.984 INFO:teuthology.orchestra.run.smithi148.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2024-03-04T13:03:03.988 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/453 2024-03-04T13:03:03.988 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/453 2024-03-04T13:03:03.988 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:04.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:04.004 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-03-04T13:03:04.186 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:04.190 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:04.190 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:04.191 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:04.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:04.412 INFO:teuthology.orchestra.run.smithi148.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2024-03-04T13:03:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/454 2024-03-04T13:03:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/454 2024-03-04T13:03:04.416 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:04.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:04.438 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-03-04T13:03:04.620 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:04.623 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:04.624 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:04.625 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:04.915 INFO:teuthology.orchestra.run.smithi148.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2024-03-04T13:03:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/455 2024-03-04T13:03:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/455 2024-03-04T13:03:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:04.921 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:04.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:04.935 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-03-04T13:03:05.114 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:05.116 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:05.118 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:05.118 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:05.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:05.354 INFO:teuthology.orchestra.run.smithi148.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T13:03:05.358 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/456 2024-03-04T13:03:05.358 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/456 2024-03-04T13:03:05.358 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:05.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:05.378 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-03-04T13:03:05.565 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:05.569 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:05.569 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:05.570 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:05.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:06.009 INFO:teuthology.orchestra.run.smithi148.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T13:03:06.014 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/457 2024-03-04T13:03:06.014 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/457 2024-03-04T13:03:06.015 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:06.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:06.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:06.224 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-03-04T13:03:06.437 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:06.440 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:06.441 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:06.442 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:06.442 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:06.790 INFO:teuthology.orchestra.run.smithi148.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T13:03:06.795 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/458 2024-03-04T13:03:06.795 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/458 2024-03-04T13:03:06.795 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:06.799 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:07.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:07.038 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-03-04T13:03:07.287 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:07.291 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:07.291 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:07.292 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:07.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:07.916 INFO:teuthology.orchestra.run.smithi148.stdout:generic/459 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T13:03:07.919 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/459 2024-03-04T13:03:07.920 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/459 2024-03-04T13:03:07.920 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:07.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:07.936 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-03-04T13:03:08.116 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:08.119 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:08.120 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:08.121 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:09.809 INFO:teuthology.orchestra.run.smithi148.stdout:generic/460 1s 2024-03-04T13:03:09.814 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/460 2024-03-04T13:03:09.815 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:09.817 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:09.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:09.825 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-03-04T13:03:10.004 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:10.007 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:10.008 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:10.009 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:10.009 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:10.449 INFO:teuthology.orchestra.run.smithi148.stdout:generic/461 [not run] ceph does not support shutdown 2024-03-04T13:03:10.453 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/461 2024-03-04T13:03:10.453 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/461 2024-03-04T13:03:10.453 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:10.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:10.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:10.469 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-03-04T13:03:10.647 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:10.651 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:10.651 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:10.652 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:10.931 INFO:teuthology.orchestra.run.smithi148.stdout:generic/462 [not run] mount 172.21.15.148:6789:/scratch with dax failed 2024-03-04T13:03:10.935 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/462 2024-03-04T13:03:10.935 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/462 2024-03-04T13:03:10.935 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:10.937 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:10.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:10.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-03-04T13:03:11.142 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:11.146 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:11.146 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:11.147 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:11.398 INFO:teuthology.orchestra.run.smithi148.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2024-03-04T13:03:11.401 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/463 2024-03-04T13:03:11.402 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/463 2024-03-04T13:03:11.402 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:03:11.404 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:03:11.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:03:11.462 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-03-04T13:03:11.862 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:03:11.866 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:03:11.866 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:03:11.867 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:03:11.867 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:07.515 INFO:teuthology.orchestra.run.smithi148.stdout:generic/464 56s 2024-03-04T13:04:07.519 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/464 2024-03-04T13:04:07.519 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:07.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:07.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:07.530 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-03-04T13:04:07.710 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:07.713 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:07.714 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:07.715 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:11.780 INFO:teuthology.orchestra.run.smithi148.stdout:generic/465 3s 2024-03-04T13:04:11.793 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/465 2024-03-04T13:04:11.793 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:11.796 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:11.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:11.812 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-03-04T13:04:12.252 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:12.258 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:12.258 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:12.260 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:12.260 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:12.475 INFO:teuthology.orchestra.run.smithi148.stdout:generic/466 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T13:04:12.479 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/466 2024-03-04T13:04:12.479 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/466 2024-03-04T13:04:12.480 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:12.482 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:12.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:12.495 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-03-04T13:04:12.669 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:12.673 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:12.673 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:12.674 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:12.674 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:23.283 INFO:teuthology.orchestra.run.smithi148.stdout:generic/467 11s 2024-03-04T13:04:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/467 2024-03-04T13:04:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:23.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:23.666 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-03-04T13:04:25.846 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:25.849 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:25.850 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:25.851 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:25.851 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:26.258 INFO:teuthology.orchestra.run.smithi148.stdout:generic/468 [not run] ceph does not support shutdown 2024-03-04T13:04:26.267 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/468 2024-03-04T13:04:26.267 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/468 2024-03-04T13:04:26.267 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:26.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:26.281 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-03-04T13:04:26.482 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:26.486 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:26.486 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:26.487 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:26.724 INFO:teuthology.orchestra.run.smithi148.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T13:04:26.728 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/469 2024-03-04T13:04:26.728 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/469 2024-03-04T13:04:26.728 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:26.730 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:26.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:26.749 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-03-04T13:04:26.930 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:26.934 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:26.934 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:26.935 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:27.271 INFO:teuthology.orchestra.run.smithi148.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2024-03-04T13:04:27.274 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/470 2024-03-04T13:04:27.274 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/470 2024-03-04T13:04:27.275 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:04:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:04:27.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:04:27.293 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-03-04T13:04:27.474 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:04:27.478 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:04:27.478 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:04:27.479 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:04:27.479 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:08.671 INFO:teuthology.orchestra.run.smithi148.stdout:generic/471 41s 2024-03-04T13:05:08.681 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/471 2024-03-04T13:05:08.681 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:05:08.684 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:08.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:05:08.752 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-03-04T13:05:09.403 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:05:09.410 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:05:09.411 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:05:09.413 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:05:09.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:09.645 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T13:05:09.638+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 574107) UID: 0 2024-03-04T13:05:09.646 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T13:05:09.639+0000 7fe45f924640 -1 Fail to open '/proc/574107/cmdline' error = (2) No such file or directory 2024-03-04T13:05:09.646 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T13:05:09.639+0000 7fe45f924640 -1 received signal: Hangup from (PID: 574107) UID: 0 2024-03-04T13:05:09.646 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T13:05:09.638+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 574107) UID: 0 2024-03-04T13:05:09.646 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T13:05:09.638+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 574107) UID: 0 2024-03-04T13:05:10.295 INFO:teuthology.orchestra.run.smithi148.stdout:generic/472 [not run] swapfiles are not supported 2024-03-04T13:05:10.301 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/472 2024-03-04T13:05:10.301 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/472 2024-03-04T13:05:10.301 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:05:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:10.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:05:10.334 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-03-04T13:05:10.517 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:05:10.520 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:05:10.521 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:05:10.522 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:05:10.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:10.792 INFO:teuthology.orchestra.run.smithi148.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-03-04T13:05:10.795 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/473 2024-03-04T13:05:10.795 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/473 2024-03-04T13:05:10.796 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:05:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:10.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:05:10.815 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-03-04T13:05:11.000 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:05:11.004 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:05:11.004 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:05:11.005 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:05:11.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:11.283 INFO:teuthology.orchestra.run.smithi148.stdout:generic/474 [not run] ceph does not support shutdown 2024-03-04T13:05:11.291 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/474 2024-03-04T13:05:11.291 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/474 2024-03-04T13:05:11.291 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:05:11.293 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:11.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:05:11.306 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-03-04T13:05:11.485 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:05:11.488 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:05:11.489 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:05:11.489 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:05:11.490 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:11.707 INFO:teuthology.orchestra.run.smithi148.stdout:generic/475 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T13:05:11.709 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/475 2024-03-04T13:05:11.709 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/475 2024-03-04T13:05:11.709 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T13:05:11.711 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:05:11.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T13:05:11.725 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-03-04T13:05:11.903 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T13:05:11.907 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T13:05:11.907 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T13:05:11.908 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T13:05:11.908 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T13:14:46.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:14:46.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:14:47.332 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T13:14:47.333+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50123) UID: 0 2024-03-04T13:14:47.341 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T13:14:47.336+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50123) UID: 0 2024-03-04T13:14:47.342 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T13:14:47.333+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50123) UID: 0 2024-03-04T13:14:47.342 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T13:14:47.337+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50123) UID: 0 2024-03-04T13:14:47.705 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T13:14:47.707+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576503) UID: 0 2024-03-04T13:14:47.706 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T13:14:47.707+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576503) UID: 0 2024-03-04T13:14:47.706 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T13:14:47.707+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576503) UID: 0 2024-03-04T13:14:47.706 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T13:14:47.707+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576503) UID: 0 2024-03-04T13:28:03.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:28:03.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:28:03.370 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T13:28:03.376+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50220) UID: 0 2024-03-04T13:28:03.370 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T13:28:03.377+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50220) UID: 0 2024-03-04T13:28:03.371 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T13:28:03.377+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50220) UID: 0 2024-03-04T13:28:03.371 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T13:28:03.377+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50220) UID: 0 2024-03-04T13:28:04.140 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T13:28:04.142+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 576713) UID: 0 2024-03-04T13:28:04.140 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T13:28:04.142+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 576713) UID: 0 2024-03-04T13:38:13.125 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T13:38:13.133+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576792) UID: 0 2024-03-04T13:38:13.126 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T13:38:13.133+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576792) UID: 0 2024-03-04T13:38:13.126 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T13:38:13.133+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576792) UID: 0 2024-03-04T13:38:13.127 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T13:38:13.133+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576792) UID: 0 2024-03-04T13:57:23.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:57:23.011 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T13:57:23.283 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T13:57:23.281+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50501) UID: 0 2024-03-04T13:57:23.290 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T13:57:23.281+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50501) UID: 0 2024-03-04T13:57:23.290 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T13:57:23.281+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50501) UID: 0 2024-03-04T13:57:23.306 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T13:57:23.297+0000 7f1dfcccf640 -1 Fail to open '/proc/50501/cmdline' error = (2) No such file or directory 2024-03-04T13:57:23.306 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T13:57:23.297+0000 7f1dfcccf640 -1 received signal: Hangup from (PID: 50501) UID: 0 2024-03-04T13:57:24.012 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T13:57:24.010+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 577012) UID: 0 2024-03-04T13:57:24.012 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T13:57:24.010+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 577012) UID: 0 2024-03-04T14:11:27.632 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T14:11:27.627+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 577153) UID: 0 2024-03-04T14:11:27.633 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T14:11:27.627+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 577153) UID: 0 2024-03-04T14:11:27.633 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T14:11:27.627+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 577153) UID: 0 2024-03-04T14:11:27.633 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T14:11:27.627+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 577153) UID: 0 2024-03-04T14:21:53.716 INFO:teuthology.orchestra.run.smithi148.stdout:generic/476 4601s 2024-03-04T14:21:53.823 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/476 2024-03-04T14:21:53.823 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:21:53.825 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:21:53.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:21:53.877 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-03-04T14:21:59.681 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:21:59.685 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:21:59.686 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:21:59.687 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:21:59.687 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:39.030 INFO:teuthology.orchestra.run.smithi148.stdout:generic/477 29s 2024-03-04T14:28:39.251 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/477 2024-03-04T14:28:39.252 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:39.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:39.461 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-03-04T14:28:41.877 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:41.882 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:41.882 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:41.883 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:52.724 INFO:teuthology.orchestra.run.smithi148.stdout:generic/478 10s 2024-03-04T14:28:52.729 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/478 2024-03-04T14:28:52.729 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:52.732 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:52.760 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:52.760 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-03-04T14:28:53.047 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:53.645 INFO:teuthology.orchestra.run.smithi148.stdout:generic/479 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:28:53.650 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/479 2024-03-04T14:28:53.650 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/479 2024-03-04T14:28:53.650 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:53.653 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:53.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:53.690 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-03-04T14:28:53.962 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:53.968 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:53.968 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:53.970 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:54.290 INFO:teuthology.orchestra.run.smithi148.stdout:generic/480 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:28:54.295 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/480 2024-03-04T14:28:54.295 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/480 2024-03-04T14:28:54.295 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:54.298 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:54.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:54.328 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-03-04T14:28:54.600 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:54.605 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:54.606 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:54.607 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:generic/481 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:28:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/481 2024-03-04T14:28:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/481 2024-03-04T14:28:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:54.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:54.951 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-03-04T14:28:55.211 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:55.215 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:55.216 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:55.220 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:55.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:55.593 INFO:teuthology.orchestra.run.smithi148.stdout:generic/482 [not run] This test requires a valid $LOGWRITES_DEV 2024-03-04T14:28:55.598 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/482 2024-03-04T14:28:55.599 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/482 2024-03-04T14:28:55.599 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:55.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:55.619 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-03-04T14:28:55.884 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:55.890 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:55.890 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:55.891 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:55.891 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:56.237 INFO:teuthology.orchestra.run.smithi148.stdout:generic/483 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:28:56.243 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/483 2024-03-04T14:28:56.243 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/483 2024-03-04T14:28:56.243 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:56.246 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:56.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:56.270 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-03-04T14:28:56.524 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:56.529 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:56.529 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:56.531 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:56.885 INFO:teuthology.orchestra.run.smithi148.stdout:generic/484 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:28:56.891 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/484 2024-03-04T14:28:56.891 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/484 2024-03-04T14:28:56.891 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:56.894 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:56.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:56.917 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-03-04T14:28:57.179 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:57.185 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:57.185 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:57.186 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:57.186 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:57.522 INFO:teuthology.orchestra.run.smithi148.stdout:generic/485 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T14:28:57.528 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/485 2024-03-04T14:28:57.528 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/485 2024-03-04T14:28:57.528 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:57.531 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:57.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:57.561 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-03-04T14:28:57.829 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:57.840 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:57.840 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:57.842 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:57.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:58.435 INFO:teuthology.orchestra.run.smithi148.stdout:generic/486 1s 2024-03-04T14:28:58.445 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/486 2024-03-04T14:28:58.445 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:58.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:58.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:58.460 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-03-04T14:28:58.695 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:58.700 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:58.700 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:58.701 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:58.701 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:59.041 INFO:teuthology.orchestra.run.smithi148.stdout:generic/487 [not run] This test requires a valid $SCRATCH_LOGDEV 2024-03-04T14:28:59.047 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/487 2024-03-04T14:28:59.048 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/487 2024-03-04T14:28:59.048 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:59.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:59.078 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-03-04T14:28:59.356 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:28:59.363 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:28:59.363 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:28:59.365 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:28:59.365 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:59.704 INFO:teuthology.orchestra.run.smithi148.stdout:generic/488 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-03-04T14:28:59.709 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/488 2024-03-04T14:28:59.709 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/488 2024-03-04T14:28:59.709 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:28:59.712 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:28:59.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:28:59.744 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-03-04T14:29:00.003 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:00.008 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:00.009 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:00.010 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:00.011 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout:generic/489 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:00.324 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/489 2024-03-04T14:29:00.324 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/489 2024-03-04T14:29:00.324 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:00.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:00.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:00.353 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-03-04T14:29:00.615 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:00.620 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:00.621 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:00.622 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:00.622 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:02.443 INFO:teuthology.orchestra.run.smithi148.stdout:generic/490 2s 2024-03-04T14:29:02.452 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/490 2024-03-04T14:29:02.452 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:02.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:02.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:02.550 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-03-04T14:29:03.004 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:03.011 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:03.011 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:03.013 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:03.013 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:03.780 INFO:teuthology.orchestra.run.smithi148.stdout:generic/491 [not run] ceph does not support freezing 2024-03-04T14:29:03.785 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/491 2024-03-04T14:29:03.785 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/491 2024-03-04T14:29:03.785 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:03.790 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:03.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:03.810 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-03-04T14:29:04.089 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:04.097 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:04.097 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:04.099 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:04.405 INFO:teuthology.orchestra.run.smithi148.stdout:generic/492 [not run] xfs_io label failed (old kernel/wrong fs?) 2024-03-04T14:29:04.410 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/492 2024-03-04T14:29:04.411 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/492 2024-03-04T14:29:04.411 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:04.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:04.432 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-03-04T14:29:04.707 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:04.713 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:04.713 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:04.714 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:04.714 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:05.383 INFO:teuthology.orchestra.run.smithi148.stdout:generic/493 [not run] swapfiles are not supported 2024-03-04T14:29:05.389 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/493 2024-03-04T14:29:05.389 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/493 2024-03-04T14:29:05.389 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:05.392 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:05.420 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:05.420 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-03-04T14:29:05.713 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:05.717 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:05.718 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:05.719 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:05.719 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:06.213 INFO:teuthology.orchestra.run.smithi148.stdout:generic/494 [not run] swapfiles are not supported 2024-03-04T14:29:06.219 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/494 2024-03-04T14:29:06.219 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/494 2024-03-04T14:29:06.219 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:06.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:06.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:06.250 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-03-04T14:29:06.520 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:06.526 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:06.526 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:06.528 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:06.528 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:07.020 INFO:teuthology.orchestra.run.smithi148.stdout:generic/495 [not run] swapfiles are not supported 2024-03-04T14:29:07.026 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/495 2024-03-04T14:29:07.026 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/495 2024-03-04T14:29:07.026 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:07.034 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:07.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:07.059 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-03-04T14:29:07.321 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:07.331 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:07.331 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:07.332 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:07.805 INFO:teuthology.orchestra.run.smithi148.stdout:generic/496 [not run] swapfiles are not supported 2024-03-04T14:29:07.812 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/496 2024-03-04T14:29:07.812 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/496 2024-03-04T14:29:07.812 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:07.816 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:07.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:07.840 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-03-04T14:29:08.109 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:08.115 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:08.115 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:08.117 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:08.582 INFO:teuthology.orchestra.run.smithi148.stdout:generic/497 [not run] swapfiles are not supported 2024-03-04T14:29:08.586 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/497 2024-03-04T14:29:08.587 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/497 2024-03-04T14:29:08.587 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:08.590 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:08.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:08.615 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-03-04T14:29:08.887 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:08.893 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:08.894 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:08.899 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:09.232 INFO:teuthology.orchestra.run.smithi148.stdout:generic/498 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:09.238 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/498 2024-03-04T14:29:09.238 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/498 2024-03-04T14:29:09.238 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:09.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:09.274 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-03-04T14:29:09.545 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:09.550 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:09.551 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:09.552 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:09.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:09.884 INFO:teuthology.orchestra.run.smithi148.stdout:generic/499 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T14:29:09.889 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/499 2024-03-04T14:29:09.889 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/499 2024-03-04T14:29:09.889 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:09.893 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:09.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:09.925 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-03-04T14:29:10.190 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:10.195 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:10.196 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:10.198 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:10.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:10.541 INFO:teuthology.orchestra.run.smithi148.stdout:generic/500 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:10.545 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/500 2024-03-04T14:29:10.545 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/500 2024-03-04T14:29:10.545 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:10.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:10.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:10.579 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-03-04T14:29:10.875 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:10.883 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:10.883 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:10.883 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:11.376 INFO:teuthology.orchestra.run.smithi148.stdout:generic/501 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T14:29:11.383 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/501 2024-03-04T14:29:11.383 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/501 2024-03-04T14:29:11.384 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:11.385 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:11.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:11.667 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-03-04T14:29:11.982 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:11.987 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:11.988 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:11.989 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:11.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:12.370 INFO:teuthology.orchestra.run.smithi148.stdout:generic/502 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:12.376 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/502 2024-03-04T14:29:12.376 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/502 2024-03-04T14:29:12.376 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:12.379 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:12.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:12.417 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-03-04T14:29:12.678 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:12.686 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:12.688 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:12.689 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:12.690 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:13.022 INFO:teuthology.orchestra.run.smithi148.stdout:generic/503 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T14:29:13.026 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/503 2024-03-04T14:29:13.026 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/503 2024-03-04T14:29:13.026 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:13.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:13.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:13.057 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-03-04T14:29:13.321 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:13.326 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:13.328 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:13.333 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:13.736 INFO:teuthology.orchestra.run.smithi148.stdout:generic/504 0s 2024-03-04T14:29:13.744 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/504 2024-03-04T14:29:13.744 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:13.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:13.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:13.780 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-03-04T14:29:14.064 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:14.068 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:14.069 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:14.071 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:14.071 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:14.618 INFO:teuthology.orchestra.run.smithi148.stdout:generic/505 [not run] ceph does not support shutdown 2024-03-04T14:29:14.642 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/505 2024-03-04T14:29:14.642 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/505 2024-03-04T14:29:14.642 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:14.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:14.662 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-03-04T14:29:14.918 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:14.928 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:14.928 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:14.930 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:15.257 INFO:teuthology.orchestra.run.smithi148.stdout:generic/506 [not run] disk quotas not supported by this filesystem type: ceph 2024-03-04T14:29:15.262 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/506 2024-03-04T14:29:15.262 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/506 2024-03-04T14:29:15.262 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:15.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:15.285 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-03-04T14:29:15.541 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:15.548 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:15.549 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:15.550 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:15.550 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:15.903 INFO:teuthology.orchestra.run.smithi148.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-03-04T14:29:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/507 2024-03-04T14:29:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/507 2024-03-04T14:29:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:15.912 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:15.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:15.953 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-03-04T14:29:16.225 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:16.229 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:16.230 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:16.231 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:16.231 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:16.530 INFO:teuthology.orchestra.run.smithi148.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph 2024-03-04T14:29:16.535 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/508 2024-03-04T14:29:16.535 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/508 2024-03-04T14:29:16.535 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:16.539 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:16.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:16.567 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-03-04T14:29:16.838 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:16.844 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:16.844 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:16.845 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:16.845 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:17.192 INFO:teuthology.orchestra.run.smithi148.stdout:generic/509 [not run] O_TMPFILE is not supported 2024-03-04T14:29:17.197 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/509 2024-03-04T14:29:17.197 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/509 2024-03-04T14:29:17.197 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:17.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:17.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:17.228 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-03-04T14:29:17.508 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:17.514 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:17.514 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:17.515 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:17.829 INFO:teuthology.orchestra.run.smithi148.stdout:generic/510 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:17.834 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/510 2024-03-04T14:29:17.835 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/510 2024-03-04T14:29:17.835 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:17.837 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:17.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:17.869 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-03-04T14:29:18.139 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:18.146 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:18.147 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:18.148 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:18.148 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:18.474 INFO:teuthology.orchestra.run.smithi148.stdout:generic/511 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-03-04T14:29:18.479 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/511 2024-03-04T14:29:18.479 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/511 2024-03-04T14:29:18.479 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:18.484 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:18.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:18.513 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-03-04T14:29:18.769 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:18.774 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:18.774 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:18.776 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:19.112 INFO:teuthology.orchestra.run.smithi148.stdout:generic/512 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-03-04T14:29:19.116 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/512 2024-03-04T14:29:19.117 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/512 2024-03-04T14:29:19.117 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:19.119 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:19.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:19.147 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-03-04T14:29:19.407 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:19.417 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:19.417 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:19.418 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:19.418 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:19.902 INFO:teuthology.orchestra.run.smithi148.stdout:generic/513 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T14:29:19.909 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/513 2024-03-04T14:29:19.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/513 2024-03-04T14:29:19.910 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:19.912 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:20.050 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:20.050 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-03-04T14:29:20.402 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:20.407 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:20.407 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:20.408 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:20.408 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:20.946 INFO:teuthology.orchestra.run.smithi148.stdout:generic/514 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T14:29:20.952 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/514 2024-03-04T14:29:20.952 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/514 2024-03-04T14:29:20.952 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:21.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:21.245 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-03-04T14:29:21.511 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:21.517 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:21.517 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:21.518 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:21.518 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:22.090 INFO:teuthology.orchestra.run.smithi148.stdout:generic/515 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T14:29:22.097 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/515 2024-03-04T14:29:22.097 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/515 2024-03-04T14:29:22.097 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:22.101 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:22.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:22.368 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-03-04T14:29:22.633 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:22.638 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:22.638 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:22.639 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:23.003 INFO:teuthology.orchestra.run.smithi148.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph 2024-03-04T14:29:23.007 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/516 2024-03-04T14:29:23.008 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/516 2024-03-04T14:29:23.008 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:23.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:23.073 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-03-04T14:29:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:24.256 INFO:teuthology.orchestra.run.smithi148.stdout:generic/517 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-03-04T14:29:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/517 2024-03-04T14:29:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/517 2024-03-04T14:29:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:24.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:24.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:24.401 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-03-04T14:29:24.768 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:24.775 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:24.775 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:25.279 INFO:teuthology.orchestra.run.smithi148.stdout:generic/518 [not run] Reflink not supported by scratch filesystem type: ceph 2024-03-04T14:29:25.288 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/518 2024-03-04T14:29:25.288 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/518 2024-03-04T14:29:25.288 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:25.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:25.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:25.494 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-03-04T14:29:25.802 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:25.807 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:25.808 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:25.809 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:26.257 INFO:teuthology.orchestra.run.smithi148.stdout:generic/519 [not run] FIBMAP not supported by this filesystem 2024-03-04T14:29:26.262 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/519 2024-03-04T14:29:26.263 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/519 2024-03-04T14:29:26.263 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:26.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:26.325 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-03-04T14:29:26.761 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:26.767 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:26.768 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:26.769 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:26.769 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:generic/520 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T14:29:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/520 2024-03-04T14:29:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/520 2024-03-04T14:29:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T14:29:27.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:29:27.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T14:29:27.224 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-03-04T14:29:27.460 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T14:29:27.470 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T14:29:27.470 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T14:29:27.471 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T14:29:27.471 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T14:30:23.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T14:30:23.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T14:30:23.629 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T14:30:23.628+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 599104) UID: 0 2024-03-04T14:30:23.629 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T14:30:23.628+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 599104) UID: 0 2024-03-04T14:30:25.490 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T14:30:25.490+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50867) UID: 0 2024-03-04T14:30:25.490 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T14:30:25.490+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50867) UID: 0 2024-03-04T14:30:25.490 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T14:30:25.490+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50867) UID: 0 2024-03-04T14:30:25.491 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T14:30:25.490+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50867) UID: 0 2024-03-04T14:44:48.912 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T14:44:48.910+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599114) UID: 0 2024-03-04T14:44:48.913 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T14:44:48.911+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599114) UID: 0 2024-03-04T14:44:48.913 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T14:44:48.910+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599114) UID: 0 2024-03-04T14:44:48.913 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T14:44:48.911+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599114) UID: 0 2024-03-04T14:54:49.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T14:54:49.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T14:54:49.876 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T14:54:49.874+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599143) UID: 0 2024-03-04T14:54:49.876 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T14:54:49.874+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599143) UID: 0 2024-03-04T14:54:49.876 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T14:54:49.874+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599143) UID: 0 2024-03-04T14:54:49.877 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T14:54:49.874+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599143) UID: 0 2024-03-04T14:54:49.957 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T14:54:49.948+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50902) UID: 0 2024-03-04T14:54:49.958 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T14:54:49.948+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50902) UID: 0 2024-03-04T14:54:49.958 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T14:54:49.949+0000 7f382c788640 -1 Fail to open '/proc/50902/cmdline' error = (2) No such file or directory 2024-03-04T14:54:49.958 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T14:54:49.949+0000 7f382c788640 -1 received signal: Hangup from (PID: 50902) UID: 0 2024-03-04T14:54:49.958 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T14:54:49.948+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50902) UID: 0 2024-03-04T15:16:32.456 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:16:32.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:16:32.696 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T15:16:32.695+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50952) UID: 0 2024-03-04T15:16:32.697 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:16:32.695+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50952) UID: 0 2024-03-04T15:16:32.697 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:16:32.695+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50952) UID: 0 2024-03-04T15:16:32.697 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T15:16:32.695+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50952) UID: 0 2024-03-04T15:16:32.769 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T15:16:32.768+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599201) UID: 0 2024-03-04T15:16:32.770 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T15:16:32.768+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599201) UID: 0 2024-03-04T15:16:32.770 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T15:16:32.768+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599201) UID: 0 2024-03-04T15:16:32.770 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T15:16:32.768+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599201) UID: 0 2024-03-04T15:21:19.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:21:19.318 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:21:49.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:21:49.391 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:22:19.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:22:19.466 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:22:49.533 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:22:49.537 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:23:19.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:23:19.965 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:23:50.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:23:50.030 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:24:20.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:24:20.128 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:24:50.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:24:50.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:25:20.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:25:20.564 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:25:50.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:25:50.638 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:26:20.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:26:20.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:26:50.841 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:26:50.844 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:27:21.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:27:21.065 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:27:51.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:27:51.137 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:28:21.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:28:21.205 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:28:21.243 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T15:28:21.241+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51233) UID: 0 2024-03-04T15:28:21.243 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T15:28:21.242+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51233) UID: 0 2024-03-04T15:28:21.244 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:28:21.242+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51233) UID: 0 2024-03-04T15:28:21.244 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:28:21.241+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51233) UID: 0 2024-03-04T15:32:12.585 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:32:12.588 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:32:12.657 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T15:32:12.655+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599502) UID: 0 2024-03-04T15:32:12.657 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T15:32:12.656+0000 7fea35801640 -1 received signal: Hangup from (PID: 599502) UID: 0 2024-03-04T15:32:12.658 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T15:32:12.655+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599502) UID: 0 2024-03-04T15:32:12.659 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T15:32:12.657+0000 7fb0f0776640 -1 Fail to open '/proc/599502/cmdline' error = (2) No such file or directory 2024-03-04T15:32:12.659 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T15:32:12.657+0000 7fb0f0776640 -1 received signal: Hangup from (PID: 599502) UID: 0 2024-03-04T15:36:09.871 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:36:09.874 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:36:10.016 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T15:36:10.015+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51275) UID: 0 2024-03-04T15:36:10.017 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:36:10.016+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51275) UID: 0 2024-03-04T15:36:10.017 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T15:36:10.016+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51275) UID: 0 2024-03-04T15:36:10.017 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:36:10.016+0000 7f1dfcccf640 -1 received signal: Hangup from (PID: 51275) UID: 0 2024-03-04T15:36:10.186 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T15:36:10.184+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599526) UID: 0 2024-03-04T15:36:10.186 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T15:36:10.184+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599526) UID: 0 2024-03-04T15:36:10.186 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T15:36:10.184+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599526) UID: 0 2024-03-04T15:36:10.187 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T15:36:10.184+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599526) UID: 0 2024-03-04T15:43:57.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:43:57.144 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:43:57.177 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T15:43:57.175+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599555) UID: 0 2024-03-04T15:43:57.177 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T15:43:57.175+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599555) UID: 0 2024-03-04T15:43:57.178 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T15:43:57.175+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599555) UID: 0 2024-03-04T15:43:57.209 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T15:43:57.175+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 599555) UID: 0 2024-03-04T15:43:58.520 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T15:43:58.518+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51303) UID: 0 2024-03-04T15:43:58.520 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T15:43:58.518+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51303) UID: 0 2024-03-04T15:43:58.520 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:43:58.518+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51303) UID: 0 2024-03-04T15:43:58.521 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:43:58.518+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51303) UID: 0 2024-03-04T15:48:27.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:48:27.373 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:48:59.319 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:48:59.322 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:49:29.384 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:49:29.388 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:49:29.622 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T15:49:29.619+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 599616) UID: 0 2024-03-04T15:49:29.622 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T15:49:29.619+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 599616) UID: 0 2024-03-04T15:53:06.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:53:06.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:53:21.203 INFO:teuthology.orchestra.run.smithi148.stdout:generic/521 5033s 2024-03-04T15:53:21.462 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/521 2024-03-04T15:53:21.462 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T15:53:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T15:53:21.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T15:53:21.975 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-03-04T15:53:22.582 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T15:53:22.586 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T15:53:22.586 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T15:53:22.587 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T15:53:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T15:53:36.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:53:36.442 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:54:06.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:54:06.829 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:54:37.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:54:37.137 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:54:37.299 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51439) UID: 0 2024-03-04T15:54:37.299 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51439) UID: 0 2024-03-04T15:54:37.299 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f1dfcccf640 -1 Fail to open '/proc/51439/cmdline' error = (2) No such file or directory 2024-03-04T15:54:37.299 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f1dfcccf640 -1 received signal: Hangup from (PID: 51439) UID: 0 2024-03-04T15:54:37.300 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f6881ba1640 -1 Fail to open '/proc/51439/cmdline' error = (2) No such file or directory 2024-03-04T15:54:37.300 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T15:54:37.296+0000 7f6881ba1640 -1 received signal: Hangup from (PID: 51439) UID: 0 2024-03-04T15:58:06.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:58:06.209 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:58:36.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:58:36.366 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:59:06.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:59:06.447 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:59:36.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T15:59:36.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:00:06.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:00:06.960 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:00:37.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:00:37.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:01:07.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:01:07.679 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:01:38.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:01:38.030 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:01:38.068 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:01:38.067+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51605) UID: 0 2024-03-04T16:01:38.069 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:01:38.067+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51605) UID: 0 2024-03-04T16:01:38.069 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:01:38.067+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51605) UID: 0 2024-03-04T16:01:38.069 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:01:38.067+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51605) UID: 0 2024-03-04T16:05:22.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:05:22.932 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:05:53.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:05:53.799 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:06:24.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:06:24.153 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:06:54.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:06:54.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:07:24.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:07:24.767 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:07:55.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:07:55.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:08:25.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:08:25.673 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:08:55.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:08:55.943 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:09:26.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:09:26.246 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:09:26.848 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:09:26.848+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600501) UID: 0 2024-03-04T16:09:26.864 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:09:26.847+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600501) UID: 0 2024-03-04T16:09:26.864 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:09:26.848+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600501) UID: 0 2024-03-04T16:09:26.865 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:09:26.847+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600501) UID: 0 2024-03-04T16:13:33.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:13:33.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:13:33.746 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:13:33.745+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600527) UID: 0 2024-03-04T16:13:33.746 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:13:33.745+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600527) UID: 0 2024-03-04T16:13:33.747 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:13:33.745+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600527) UID: 0 2024-03-04T16:13:33.747 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:13:33.746+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600527) UID: 0 2024-03-04T16:13:33.758 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:13:33.757+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51801) UID: 0 2024-03-04T16:13:33.759 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:13:33.758+0000 7f382c788640 -1 Fail to open '/proc/51801/cmdline' error = (2) No such file or directory 2024-03-04T16:13:33.759 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:13:33.758+0000 7f382c788640 -1 received signal: Hangup from (PID: 51801) UID: 0 2024-03-04T16:13:33.759 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:13:33.758+0000 7f21a389d640 -1 Fail to open '/proc/51801/cmdline' error = (2) No such file or directory 2024-03-04T16:13:33.759 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:13:33.758+0000 7f21a389d640 -1 received signal: Hangup from (PID: 51801) UID: 0 2024-03-04T16:13:33.760 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:13:33.757+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51801) UID: 0 2024-03-04T16:17:36.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:17:36.712 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:17:36.993 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51824) UID: 0 2024-03-04T16:17:36.994 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51824) UID: 0 2024-03-04T16:17:36.994 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f1dfcccf640 -1 Fail to open '/proc/51824/cmdline' error = (2) No such file or directory 2024-03-04T16:17:36.994 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f1dfcccf640 -1 received signal: Hangup from (PID: 51824) UID: 0 2024-03-04T16:17:36.994 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f382c788640 -1 Fail to open '/proc/51824/cmdline' error = (2) No such file or directory 2024-03-04T16:17:36.994 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:17:36.993+0000 7f382c788640 -1 received signal: Hangup from (PID: 51824) UID: 0 2024-03-04T16:17:37.039 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:17:37.039+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600550) UID: 0 2024-03-04T16:17:37.040 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:17:37.039+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600550) UID: 0 2024-03-04T16:17:37.040 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:17:37.039+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600550) UID: 0 2024-03-04T16:17:37.041 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:17:37.039+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600550) UID: 0 2024-03-04T16:25:20.195 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:25:20.198 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:25:20.241 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:25:20.240+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51853) UID: 0 2024-03-04T16:25:20.244 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:25:20.243+0000 7f382c788640 -1 received signal: Hangup from (PID: 51853) UID: 0 2024-03-04T16:25:20.244 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:25:20.243+0000 7f6881ba1640 -1 received signal: Hangup from (PID: 51853) UID: 0 2024-03-04T16:25:20.244 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:25:20.241+0000 7f1dfcccf640 -1 received signal: Hangup from (PID: 51853) UID: 0 2024-03-04T16:29:30.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:29:30.621 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:30:00.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:30:00.879 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:30:30.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:30:31.004 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:31:01.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:31:01.141 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:31:31.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:31:31.213 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:32:01.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:32:01.502 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:32:31.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:32:31.585 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:33:02.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:33:02.011 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:33:02.057 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:33:02.064+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52008) UID: 0 2024-03-04T16:33:02.058 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:33:02.064+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52008) UID: 0 2024-03-04T16:33:02.058 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:33:02.064+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52008) UID: 0 2024-03-04T16:33:02.058 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:33:02.064+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52008) UID: 0 2024-03-04T16:36:53.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:36:53.939 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:37:24.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:37:24.277 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:37:54.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:37:54.359 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:38:24.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:38:24.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:38:24.512 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:38:24.524+0000 7f382c788640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52084) UID: 0 2024-03-04T16:38:24.512 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:38:24.524+0000 7f21a389d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52084) UID: 0 2024-03-04T16:38:24.512 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:38:24.524+0000 7f6881ba1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52084) UID: 0 2024-03-04T16:38:24.513 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:38:24.525+0000 7f1dfcccf640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52084) UID: 0 2024-03-04T16:38:24.535 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:38:24.549+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600801) UID: 0 2024-03-04T16:38:24.536 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:38:24.549+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600801) UID: 0 2024-03-04T16:38:24.536 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:38:24.549+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600801) UID: 0 2024-03-04T16:38:24.542 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:38:24.549+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 600801) UID: 0 2024-03-04T16:42:14.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:42:14.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:42:44.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:42:44.926 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:43:15.207 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:43:15.210 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:43:45.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:43:45.774 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:44:16.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:44:16.107 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:44:16.167 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T16:44:16.184+0000 7fa25d993640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 600898) UID: 0 2024-03-04T16:44:16.167 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T16:44:16.185+0000 7f9bc7e2b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 600898) UID: 0 2024-03-04T16:45:56.994 INFO:teuthology.orchestra.run.smithi148.stdout:generic/522 3154s 2024-03-04T16:45:57.128 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/522 2024-03-04T16:45:57.128 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:45:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:45:57.222 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:45:57.223 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-03-04T16:45:58.081 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:45:58.086 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:45:58.086 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:45:58.087 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:45:58.087 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:45:58.728 INFO:teuthology.orchestra.run.smithi148.stdout:generic/523 0s 2024-03-04T16:45:58.737 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/523 2024-03-04T16:45:58.737 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:45:58.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:45:58.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:45:58.755 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-03-04T16:45:59.020 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:45:59.026 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:45:59.029 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:45:59.030 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:45:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:03.339 INFO:teuthology.orchestra.run.smithi148.stdout:generic/524 4s 2024-03-04T16:46:03.350 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/524 2024-03-04T16:46:03.350 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:03.355 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:03.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:03.367 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-03-04T16:46:03.645 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:03.650 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:03.651 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:03.652 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:03.652 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:04.150 INFO:teuthology.orchestra.run.smithi148.stdout:generic/525 [not run] filesystem does not support huge file size 2024-03-04T16:46:04.167 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/525 2024-03-04T16:46:04.167 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/525 2024-03-04T16:46:04.167 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:04.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:04.199 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-03-04T16:46:04.481 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:04.489 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:04.492 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:04.493 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:04.493 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:04.971 INFO:teuthology.orchestra.run.smithi148.stdout:generic/526 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T16:46:04.976 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/526 2024-03-04T16:46:04.976 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/526 2024-03-04T16:46:04.977 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:05.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:05.008 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-03-04T16:46:05.279 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:05.284 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:05.284 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:05.286 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:05.629 INFO:teuthology.orchestra.run.smithi148.stdout:generic/527 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T16:46:05.634 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/527 2024-03-04T16:46:05.634 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/527 2024-03-04T16:46:05.634 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:05.636 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:05.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:05.667 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-03-04T16:46:05.899 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:05.904 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:05.905 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:05.908 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:05.908 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:06.342 INFO:teuthology.orchestra.run.smithi148.stdout:generic/528 0s 2024-03-04T16:46:06.353 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/528 2024-03-04T16:46:06.353 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:06.356 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:06.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:06.391 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-03-04T16:46:06.804 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:06.813 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:06.814 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:06.815 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:06.815 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:07.395 INFO:teuthology.orchestra.run.smithi148.stdout:generic/529 1s 2024-03-04T16:46:07.401 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/529 2024-03-04T16:46:07.402 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:07.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:07.415 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-03-04T16:46:07.684 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:07.689 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:07.689 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:07.691 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:08.147 INFO:teuthology.orchestra.run.smithi148.stdout:generic/530 [not run] ceph does not support shutdown 2024-03-04T16:46:08.166 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/530 2024-03-04T16:46:08.166 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/530 2024-03-04T16:46:08.166 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:08.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:08.189 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-03-04T16:46:08.456 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:08.460 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:08.461 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:08.462 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:08.462 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:08.752 INFO:teuthology.orchestra.run.smithi148.stdout:generic/531 [not run] O_TMPFILE is not supported 2024-03-04T16:46:08.757 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/531 2024-03-04T16:46:08.757 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/531 2024-03-04T16:46:08.757 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:08.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:08.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:08.786 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-03-04T16:46:09.059 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:09.063 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:09.063 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:09.065 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:09.065 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:09.425 INFO:teuthology.orchestra.run.smithi148.stdout:generic/532 0s 2024-03-04T16:46:09.437 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/532 2024-03-04T16:46:09.437 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:09.441 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:09.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:09.460 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-03-04T16:46:09.721 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:09.727 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:09.728 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:09.729 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:09.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:10.628 INFO:teuthology.orchestra.run.smithi148.stdout:generic/533 1s 2024-03-04T16:46:10.642 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/533 2024-03-04T16:46:10.642 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:10.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:10.665 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-03-04T16:46:10.921 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:10.927 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:10.928 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:10.929 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:10.929 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:11.308 INFO:teuthology.orchestra.run.smithi148.stdout:generic/534 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T16:46:11.314 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/534 2024-03-04T16:46:11.314 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/534 2024-03-04T16:46:11.314 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:11.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:11.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:11.347 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-03-04T16:46:11.584 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:11.589 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:11.589 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:11.590 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:11.590 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:11.921 INFO:teuthology.orchestra.run.smithi148.stdout:generic/535 [not run] require 172.21.15.148:6789:/scratch to be valid block disk 2024-03-04T16:46:11.926 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/535 2024-03-04T16:46:11.926 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/535 2024-03-04T16:46:11.926 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:11.929 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:11.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:11.952 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-03-04T16:46:12.191 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:12.195 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:12.196 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:12.197 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:12.197 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:12.580 INFO:teuthology.orchestra.run.smithi148.stdout:generic/536 [not run] ceph does not support shutdown 2024-03-04T16:46:12.592 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/536 2024-03-04T16:46:12.592 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/536 2024-03-04T16:46:12.592 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:12.594 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:12.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:12.618 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-03-04T16:46:12.875 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:12.880 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:12.880 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:12.884 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:12.884 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:13.534 INFO:teuthology.orchestra.run.smithi148.stdout:generic/537 [not run] FSTRIM not supported 2024-03-04T16:46:13.556 INFO:teuthology.orchestra.run.smithi148.stdout:Ran: generic/537 2024-03-04T16:46:13.556 INFO:teuthology.orchestra.run.smithi148.stdout:Not run: generic/537 2024-03-04T16:46:13.557 INFO:teuthology.orchestra.run.smithi148.stdout:Passed all 1 tests 2024-03-04T16:46:13.559 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:46:13.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-03-04T16:46:13.579 DEBUG:teuthology.orchestra.run.smithi148:> (cd /tmp/tmp.ATjWjXU3R3xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-03-04T16:46:13.825 INFO:teuthology.orchestra.run.smithi148.stdout:FSTYP -- ceph 2024-03-04T16:46:13.829 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM -- Linux/x86_64 smithi148 5.14.0-427.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024 2024-03-04T16:46:13.830 INFO:teuthology.orchestra.run.smithi148.stdout:MKFS_OPTIONS -- 172.21.15.148:6789:/scratch 2024-03-04T16:46:13.832 INFO:teuthology.orchestra.run.smithi148.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBQuuVl4yA1CxAAk8oFLh6e1PLuTBMRyuRN2w== -o context=system_u:object_r:root_t:s0 172.21.15.148:6789:/scratch /tmp/tmp.PKHzEB1Z0hscratch 2024-03-04T16:46:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:47:23.504 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:47:23.522+0000 7f2199ea7640 -1 bdev(0x555f6abb4e00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f2199ea7640 time 2024-03-04T16:47:23.523616+0000 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555f68102451] 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.505 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:47:23.523+0000 7f2199ea7640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f2199ea7640 time 2024-03-04T16:47:23.523616+0000 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555f68102451] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr:*** Caught signal (Aborted) ** 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: in thread 7f2199ea7640 thread_name:bstore_aio 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f21a6c54db0] 2024-03-04T16:47:23.506 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f21a6ca154c] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: raise() 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: abort() 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555f68102512] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 7: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-04T16:47:23.524+0000 7f2199ea7640 -1 *** Caught signal (Aborted) ** 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: in thread 7f2199ea7640 thread_name:bstore_aio 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f21a6c54db0] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f21a6ca154c] 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: raise() 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: abort() 2024-03-04T16:47:23.507 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555f68102512] 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: 7: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T16:47:23.508 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: -6> 2024-03-04T16:47:23.522+0000 7f2199ea7640 -1 bdev(0x555f6abb4e00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: -1> 2024-03-04T16:47:23.523+0000 7f2199ea7640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f2199ea7640 time 2024-03-04T16:47:23.523616+0000 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555f68102451] 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 0> 2024-03-04T16:47:23.524+0000 7f2199ea7640 -1 *** Caught signal (Aborted) ** 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: in thread 7f2199ea7640 thread_name:bstore_aio 2024-03-04T16:47:23.523 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f21a6c54db0] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f21a6ca154c] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: raise() 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: abort() 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555f68102512] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 7: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T16:47:23.524 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: -380> 2024-03-04T16:47:23.522+0000 7f2199ea7640 -1 bdev(0x555f6abb4e00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: -375> 2024-03-04T16:47:23.523+0000 7f2199ea7640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f2199ea7640 time 2024-03-04T16:47:23.523616+0000 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1845-g717ce3cc/rpm/el9/BUILD/ceph-19.0.0-1845-g717ce3cc/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555f68102451] 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: -374> 2024-03-04T16:47:23.524+0000 7f2199ea7640 -1 *** Caught signal (Aborted) ** 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: in thread 7f2199ea7640 thread_name:bstore_aio 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.536 INFO:tasks.ceph.osd.5.smithi191.stderr: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev) 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f21a6c54db0] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f21a6ca154c] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 3: raise() 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 4: abort() 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555f68102512] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555f68ad1f60] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 7: ceph-osd(+0xdc50d1) [0x555f68ad20d1] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f21a6c9f802] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f21a6c3f450] 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T16:47:23.537 INFO:tasks.ceph.osd.5.smithi191.stderr: 2024-03-04T16:47:23.659 INFO:tasks.ceph.osd.5.smithi191.stderr:daemon-helper: command failed with exit status 5 2024-03-04T16:47:26.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-04T16:47:28.883 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:47:28.901+0000 7fea35801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610213) UID: 0 2024-03-04T16:47:28.883 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:47:28.901+0000 7fe45f924640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610213) UID: 0 2024-03-04T16:47:28.883 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:47:28.902+0000 7f19b5ca4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610213) UID: 0 2024-03-04T16:47:28.884 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:47:28.902+0000 7fb0f0776640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610213) UID: 0 2024-03-04T16:47:33.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~7s 2024-03-04T16:47:39.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~13s 2024-03-04T16:47:46.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-03-04T16:47:52.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~26s 2024-03-04T16:47:59.572 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~33s 2024-03-04T16:48:06.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-03-04T16:48:12.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~46s 2024-03-04T16:48:19.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~53s 2024-03-04T16:48:26.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-03-04T16:48:32.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~66s 2024-03-04T16:48:39.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~73s 2024-03-04T16:48:45.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~79s 2024-03-04T16:48:52.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~86s 2024-03-04T16:48:59.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~93s 2024-03-04T16:49:05.687 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~99s 2024-03-04T16:49:12.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~106s 2024-03-04T16:49:18.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~112s 2024-03-04T16:49:25.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~119s 2024-03-04T16:49:32.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2024-03-04T16:49:38.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~132s 2024-03-04T16:49:45.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~139s 2024-03-04T16:49:51.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2024-03-04T16:49:58.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~152s 2024-03-04T16:50:05.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~159s 2024-03-04T16:50:11.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-03-04T16:50:18.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~172s 2024-03-04T16:50:25.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~179s 2024-03-04T16:50:31.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-03-04T16:50:38.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~192s 2024-03-04T16:50:44.866 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~198s 2024-03-04T16:50:51.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2024-03-04T16:50:58.091 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~212s 2024-03-04T16:51:04.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~218s 2024-03-04T16:51:11.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~225s 2024-03-04T16:51:17.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~231s 2024-03-04T16:51:24.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~238s 2024-03-04T16:51:31.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2024-03-04T16:51:37.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~251s 2024-03-04T16:51:44.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~258s 2024-03-04T16:51:50.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~264s 2024-03-04T16:51:57.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~271s 2024-03-04T16:52:04.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~278s 2024-03-04T16:52:08.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:52:08.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:52:10.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~284s 2024-03-04T16:52:17.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~291s 2024-03-04T16:52:24.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~298s 2024-03-04T16:52:30.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~304s 2024-03-04T16:52:30.652 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-03-04T16:52:30.653 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T16:52:30.653 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T16:52:31.449 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T16:52:31.450 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T16:52:31.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T16:52:32.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.603+0000 7f27bf59e640 1 Processor -- start 2024-03-04T16:52:32.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.603+0000 7f27bf59e640 1 -- start start 2024-03-04T16:52:32.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.604+0000 7f27bf59e640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f27b80ead00 0x7f27b804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27bdd9b640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f27b804ffd0 0x7f27b8037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.148:42532/0 (socket says 172.21.15.148:42532) 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27be59c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f27b80ead00 0x7f27b804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27be59c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f27b80ead00 0x7f27b804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:56232/0 (socket says 172.21.15.148:56232) 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27bdd9b640 1 -- 172.21.15.148:0/370915606 learned_addr learned my addr 172.21.15.148:0/370915606 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b8053720 con 0x7f27b8006ea0 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b8053a00 con 0x7f27b804ffd0 2024-03-04T16:52:32.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.605+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b8054330 con 0x7f27b80ead00 2024-03-04T16:52:32.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221195733 0 0) 0x7f27b8053a00 con 0x7f27b804ffd0 2024-03-04T16:52:32.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac003520 con 0x7f27b804ffd0 2024-03-04T16:52:32.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055160120 0 0) 0x7f27b8053720 con 0x7f27b8006ea0 2024-03-04T16:52:32.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8053a00 con 0x7f27b8006ea0 2024-03-04T16:52:32.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231497928 0 0) 0x7f27b8053a00 con 0x7f27b8006ea0 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 >> v2:172.21.15.191:3300/0 conn(0x7f27b80ead00 msgr2=0x7f27b804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 --2- 172.21.15.148:0/370915606 >> v2:172.21.15.191:3300/0 conn(0x7f27b80ead00 0x7f27b804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 >> v1:172.21.15.191:6789/0 conn(0x7f27b804ffd0 legacy=0x7f27b8037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8053ce0 con 0x7f27b8006ea0 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 486913995 0 0) 0x7f27c004f8e0 con 0x7f27b8006ea0 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 >> v1:172.21.15.148:6789/0 conn(0x7f27b8006ea0 legacy=0x7f27b80382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 0x7f27ac004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ac0037a0 0x7f27ac003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004160 0x7f27ac00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8054330 con 0x7f27ac004160 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac00bf80 con 0x7f27ac004ba0 2024-03-04T16:52:32.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.606+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac00c100 con 0x7f27ac0037a0 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 0x7f27ac004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27bdd9b640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ac0037a0 0x7f27ac003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 -- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ac0037a0 msgr2=0x7f27ac003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ac0037a0 0x7f27ac003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 -- 172.21.15.148:0/370915606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004160 msgr2=0x7f27ac00ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004160 0x7f27ac00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.607+0000 7f27be59c640 1 -- 172.21.15.148:0/370915606 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8053ce0 con 0x7f27ac004ba0 2024-03-04T16:52:32.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.608+0000 7f27be59c640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 0x7f27ac004fa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009ac0 tx=0x7f27ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38530f86c91c49b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.608+0000 7f27bed9d640 1 -- 172.21.15.148:0/370915606 reap_dead start 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.609+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0039070 con 0x7f27ac004ba0 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.609+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b002c710 con 0x7f27ac004ba0 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.609+0000 7f27bd59a640 1 -- 172.21.15.148:0/370915606 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0009690 con 0x7f27ac004ba0 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.610+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 msgr2=0x7f27ac004fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.610+0000 7f27bf59e640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 0x7f27ac004fa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009ac0 tx=0x7f27ac005830 comp rx=0 tx=0).stop 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.611+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 shutdown_connections 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.611+0000 7f27bf59e640 1 --2- 172.21.15.148:0/370915606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ac004ba0 0x7f27ac004fa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.611+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 >> 172.21.15.148:0/370915606 conn(0x7f27b8009fb0 msgr2=0x7f27b800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:32.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.615+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 shutdown_connections 2024-03-04T16:52:32.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.615+0000 7f27bf59e640 1 -- 172.21.15.148:0/370915606 wait complete. 2024-03-04T16:52:32.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.618+0000 7f27bf59e640 1 Processor -- start 2024-03-04T16:52:32.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.618+0000 7f27bf59e640 1 -- start start 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.619+0000 7f27bf59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004ba0 0x7f27b80df100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.619+0000 7f27bf59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.619+0000 7f27bf59e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27b8044f10 0x7f27b8045390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.619+0000 7f27bf59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8054940 con 0x7f27ac004ba0 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bdd9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bdd9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.148:56248/0 (socket says 172.21.15.148:56248) 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bdd9b640 1 -- 172.21.15.148:0/2626781984 learned_addr learned my addr 172.21.15.148:0/2626781984 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27be59c640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004ba0 0x7f27b80df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bed9d640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27b8044f10 0x7f27b8045390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80547c0 con 0x7f27b8044090 2024-03-04T16:52:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.620+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b804f550 con 0x7f27b8044f10 2024-03-04T16:52:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 -- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27b8044f10 msgr2=0x7f27b8045390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27b8044f10 0x7f27b8045390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 -- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004ba0 msgr2=0x7f27b80df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004ba0 0x7f27b80df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8047340 con 0x7f27b8044090 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27bdd9b640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f27a8001720 tx=0x7f27a8001be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8007730 con 0x7f27b8044090 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.621+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8018070 con 0x7f27b8044090 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.622+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800eb30 con 0x7f27b8044090 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.622+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b804bf70 con 0x7f27b8044090 2024-03-04T16:52:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.622+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b804c530 con 0x7f27b8044090 2024-03-04T16:52:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.625+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f27a800ecd0 con 0x7f27b8044090 2024-03-04T16:52:32.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.625+0000 7f27b6ffd640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 0x7f27a006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:32.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.625+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f27a808a190 con 0x7f27b8044090 2024-03-04T16:52:32.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.626+0000 7f27be59c640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 0x7f27a006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.626+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8005390 con 0x7f27b8044090 2024-03-04T16:52:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.627+0000 7f27be59c640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 0x7f27a006d9e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f27b0036000 tx=0x7f27b0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:32.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.635+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27a80587f0 con 0x7f27b8044090 2024-03-04T16:52:32.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.935+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27b8049120 con 0x7f27b8044090 2024-03-04T16:52:32.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.937+0000 7f27b6ffd640 1 -- 172.21.15.148:0/2626781984 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17138 (secure 0 0 0) 0x7f27a805c890 con 0x7f27b8044090 2024-03-04T16:52:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-04T16:52:32.921 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":42,"fsid":"d4b3ed31-a94c-42d0-968e-1d6a4e199a28","created":"2024-03-04T12:11:30.511971+0000","modified":"2024-03-04T16:47:24.910415+0000","last_up_change":"2024-03-04T16:47:23.901010+0000","last_in_change":"2024-03-04T12:11:32.851843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T12:11:42.758374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-04T12:13:04.288877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-04T12:13:05.037102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6100000143051147,"score_stable":1.6100000143051147,"optimal_score":0.87999999523162842,"raw_score_acting":1.4199999570846558,"raw_score_stable":1.4199999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b5cd59-dfd9-4906-8f61-b9057971ed30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6801","nonce":3066481003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6803","nonce":3066481003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6807","nonce":3066481003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3066481003},{"type":"v1","addr":"172.21.15.148:6805","nonce":3066481003}]},"public_addr":"172.21.15.148:6801/3066481003","cluster_addr":"172.21.15.148:6803/3066481003","heartbeat_back_addr":"172.21.15.148:6807/3066481003","heartbeat_front_addr":"172.21.15.148:6805/3066481003","state":["exists","up"]},{"osd":1,"uuid":"af8b755c-212e-46c7-acac-dd927ca0da2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6825","nonce":3160501011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6827","nonce":3160501011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6831","nonce":3160501011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3160501011},{"type":"v1","addr":"172.21.15.148:6829","nonce":3160501011}]},"public_addr":"172.21.15.148:6825/3160501011","cluster_addr":"172.21.15.148:6827/3160501011","heartbeat_back_addr":"172.21.15.148:6831/3160501011","heartbeat_front_addr":"172.21.15.148:6829/3160501011","state":["exists","up"]},{"osd":2,"uuid":"e967f2b6-df78-4a51-af8c-b1e26fe8b4d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6810","nonce":2881995606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6814","nonce":2881995606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6822","nonce":2881995606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2881995606},{"type":"v1","addr":"172.21.15.148:6818","nonce":2881995606}]},"public_addr":"172.21.15.148:6810/2881995606","cluster_addr":"172.21.15.148:6814/2881995606","heartbeat_back_addr":"172.21.15.148:6822/2881995606","heartbeat_front_addr":"172.21.15.148:6818/2881995606","state":["exists","up"]},{"osd":3,"uuid":"41ce11cf-66ba-44e8-a0e5-3e976505e466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6811","nonce":3796620207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6815","nonce":3796620207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6821","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6823","nonce":3796620207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":3796620207},{"type":"v1","addr":"172.21.15.148:6819","nonce":3796620207}]},"public_addr":"172.21.15.148:6811/3796620207","cluster_addr":"172.21.15.148:6815/3796620207","heartbeat_back_addr":"172.21.15.148:6823/3796620207","heartbeat_front_addr":"172.21.15.148:6819/3796620207","state":["exists","up"]},{"osd":4,"uuid":"3c655fef-c6c1-4080-8af7-217ce635c225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6801","nonce":2064455370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6803","nonce":2064455370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6807","nonce":2064455370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2064455370},{"type":"v1","addr":"172.21.15.191:6805","nonce":2064455370}]},"public_addr":"172.21.15.191:6801/2064455370","cluster_addr":"172.21.15.191:6803/2064455370","heartbeat_back_addr":"172.21.15.191:6807/2064455370","heartbeat_front_addr":"172.21.15.191:6805/2064455370","state":["exists","up"]},{"osd":5,"uuid":"4b5c175f-dfcd-43c8-b96d-c65760b94902","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":41,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6817","nonce":2924287970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6819","nonce":2924287970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6823","nonce":2924287970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2924287970},{"type":"v1","addr":"172.21.15.191:6821","nonce":2924287970}]},"public_addr":"172.21.15.191:6817/2924287970","cluster_addr":"172.21.15.191:6819/2924287970","heartbeat_back_addr":"172.21.15.191:6823/2924287970","heartbeat_front_addr":"172.21.15.191:6821/2924287970","state":["exists"]},{"osd":6,"uuid":"35c6c230-5f91-4544-a15c-b43b4021a946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6809","nonce":1171226865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6811","nonce":1171226865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6815","nonce":1171226865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1171226865},{"type":"v1","addr":"172.21.15.191:6813","nonce":1171226865}]},"public_addr":"172.21.15.191:6809/1171226865","cluster_addr":"172.21.15.191:6811/1171226865","heartbeat_back_addr":"172.21.15.191:6815/1171226865","heartbeat_front_addr":"172.21.15.191:6813/1171226865","state":["exists","up"]},{"osd":7,"uuid":"b962274a-4949-4442-a2fc-527f35e5b1d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6825","nonce":1686676970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6827","nonce":1686676970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6831","nonce":1686676970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":1686676970},{"type":"v1","addr":"172.21.15.191:6829","nonce":1686676970}]},"public_addr":"172.21.15.191:6825/1686676970","cluster_addr":"172.21.15.191:6827/1686676970","heartbeat_back_addr":"172.21.15.191:6831/1686676970","heartbeat_front_addr":"172.21.15.191:6829/1686676970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.965885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.024879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.034514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.108882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.013135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971394+0000","dead_epoch":41},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:37.971949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T12:11:38.003048+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.36","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T16:52:32.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 msgr2=0x7f27a006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 0x7f27a006d9e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f27b0036000 tx=0x7f27b0030000 comp rx=0 tx=0).stop 2024-03-04T16:52:32.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 msgr2=0x7f27b804d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:32.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f27a8001720 tx=0x7f27a8001be0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 shutdown_connections 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f27a006b520 0x7f27a006d9e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27b8044f10 0x7f27b8045390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27b8044090 0x7f27b804d010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 --2- 172.21.15.148:0/2626781984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27ac004ba0 0x7f27b80df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 >> 172.21.15.148:0/2626781984 conn(0x7f27b8009fb0 msgr2=0x7f27b803a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 shutdown_connections 2024-03-04T16:52:32.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:32.942+0000 7f27bf59e640 1 -- 172.21.15.148:0/2626781984 wait complete. 2024-03-04T16:52:32.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T12:11:42.758374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T12:13:04.288877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T12:13:05.037102+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6100000143051147, 'score_stable': 1.6100000143051147, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.4199999570846558, 'raw_score_stable': 1.4199999570846558, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T16:52:32.942 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T16:52:33.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.135+0000 7f34bffff640 1 Processor -- start 2024-03-04T16:52:33.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.136+0000 7f34bffff640 1 -- start start 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34bffff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 0x7f34c00cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34bffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34c015d5b0 0x7f34c00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34b7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34c015d5b0 0x7f34c00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34bffff640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c0159df0 con 0x7f34c006de40 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34b7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34c015d5b0 0x7f34c00cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38242/0 (socket says 172.21.15.148:38242) 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34b7fff640 1 -- 172.21.15.148:0/2849284727 learned_addr learned my addr 172.21.15.148:0/2849284727 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.137+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c015db20 con 0x7f34c015d5b0 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.138+0000 7f34beffd640 1 --2- 172.21.15.148:0/2849284727 >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 0x7f34c00cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.138+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c015de00 con 0x7f34c0159a10 2024-03-04T16:52:33.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.140+0000 7f34b7fff640 1 -- 172.21.15.148:0/2849284727 >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 msgr2=0x7f34c00cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:33.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.140+0000 7f34b7fff640 1 --2- 172.21.15.148:0/2849284727 >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 0x7f34c00cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.140+0000 7f34b7fff640 1 -- 172.21.15.148:0/2849284727 >> v1:172.21.15.191:6789/0 conn(0x7f34c006de40 legacy=0x7f34c0071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:33.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.140+0000 7f34b7fff640 1 -- 172.21.15.148:0/2849284727 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c015e0e0 con 0x7f34c015d5b0 2024-03-04T16:52:33.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.140+0000 7f34beffd640 1 --2- 172.21.15.148:0/2849284727 >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 0x7f34c00cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T16:52:33.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.142+0000 7f34b7fff640 1 --2- 172.21.15.148:0/2849284727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c00cf710 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f34c006bc50 tx=0x7f34b002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae1b750e611337b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.145+0000 7f34be7fc640 1 -- 172.21.15.148:0/2849284727 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0008df0 con 0x7f34c015d5b0 2024-03-04T16:52:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.145+0000 7f34be7fc640 1 -- 172.21.15.148:0/2849284727 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0036070 con 0x7f34c015d5b0 2024-03-04T16:52:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.145+0000 7f34be7fc640 1 -- 172.21.15.148:0/2849284727 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000a2f0 con 0x7f34c015d5b0 2024-03-04T16:52:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.146+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 msgr2=0x7f34c00cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.146+0000 7f34bffff640 1 --2- 172.21.15.148:0/2849284727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c00cf710 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f34c006bc50 tx=0x7f34b002b920 comp rx=0 tx=0).stop 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 shutdown_connections 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 --2- 172.21.15.148:0/2849284727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c00cf710 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 --2- 172.21.15.148:0/2849284727 >> v2:172.21.15.191:3300/0 conn(0x7f34c0159a10 0x7f34c00cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 >> 172.21.15.148:0/2849284727 conn(0x7f34c00670e0 msgr2=0x7f34c006ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 shutdown_connections 2024-03-04T16:52:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.147+0000 7f34bffff640 1 -- 172.21.15.148:0/2849284727 wait complete. 2024-03-04T16:52:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.148+0000 7f34bffff640 1 Processor -- start 2024-03-04T16:52:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.148+0000 7f34bffff640 1 -- start start 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c0159a10 0x7f34c015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c0159030 0x7f34c015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00cc660 con 0x7f34c015d5b0 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00b2740 con 0x7f34c0159030 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34bffff640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00cee90 con 0x7f34c0159a10 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34beffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c0159a10 0x7f34c015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38254/0 (socket says 172.21.15.148:38254) 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.149+0000 7f34b7fff640 1 -- 172.21.15.148:0/3943085906 learned_addr learned my addr 172.21.15.148:0/3943085906 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bf7fe640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c0159030 0x7f34c015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 -- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c0159a10 msgr2=0x7f34c015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c0159a10 0x7f34c015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 -- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c0159030 msgr2=0x7f34c015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c0159030 0x7f34c015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 -- 172.21.15.148:0/3943085906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c0158270 con 0x7f34c015d5b0 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34b7fff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000c00 tx=0x7f34b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0031d70 con 0x7f34c015d5b0 2024-03-04T16:52:33.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c0158550 con 0x7f34c015d5b0 2024-03-04T16:52:33.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c0143350 con 0x7f34c015d5b0 2024-03-04T16:52:33.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b004e480 con 0x7f34c015d5b0 2024-03-04T16:52:33.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.150+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b004e870 con 0x7f34c015d5b0 2024-03-04T16:52:33.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.152+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c004f090 con 0x7f34c015d5b0 2024-03-04T16:52:33.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.153+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f34b004ea10 con 0x7f34c015d5b0 2024-03-04T16:52:33.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.154+0000 7f34bcff9640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.154+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f34b00b62f0 con 0x7f34c015d5b0 2024-03-04T16:52:33.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.154+0000 7f34beffd640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.155+0000 7f34beffd640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 0x7f34a006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34a800bec0 tx=0x7f34a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.158+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34b0084ff0 con 0x7f34c015d5b0 2024-03-04T16:52:33.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.450+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34c006d710 con 0x7f34c015d5b0 2024-03-04T16:52:33.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.452+0000 7f34bcff9640 1 -- 172.21.15.148:0/3943085906 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34b00848f0 con 0x7f34c015d5b0 2024-03-04T16:52:33.432 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-04T16:52:33.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 msgr2=0x7f34a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 0x7f34a006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34a800bec0 tx=0x7f34a8008040 comp rx=0 tx=0).stop 2024-03-04T16:52:33.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 msgr2=0x7f34c0142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000c00 tx=0x7f34b00046c0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 shutdown_connections 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c0159030 0x7f34c015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c015d5b0 0x7f34c0142e10 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 --2- 172.21.15.148:0/3943085906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c0159a10 0x7f34c015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 >> 172.21.15.148:0/3943085906 conn(0x7f34c006de40 msgr2=0x7f34c00cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 shutdown_connections 2024-03-04T16:52:33.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.456+0000 7f34bffff640 1 -- 172.21.15.148:0/3943085906 wait complete. 2024-03-04T16:52:33.453 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T16:52:33.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.649+0000 7f001ecfa640 1 Processor -- start 2024-03-04T16:52:33.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.649+0000 7f001ecfa640 1 -- start start 2024-03-04T16:52:33.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ecfa640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00180b5c90 0x7f00181621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ecfa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00180b9420 0x7f00180ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ecfa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180d3510 con 0x7f001815dc60 2024-03-04T16:52:33.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ecfa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0018162720 con 0x7f00180b9420 2024-03-04T16:52:33.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ecfa640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180d19f0 con 0x7f00180b5c90 2024-03-04T16:52:33.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001d297640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00180b9420 0x7f00180ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001d297640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00180b9420 0x7f00180ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38256/0 (socket says 172.21.15.148:38256) 2024-03-04T16:52:33.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001d297640 1 -- 172.21.15.148:0/2284971140 learned_addr learned my addr 172.21.15.148:0/2284971140 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:33.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f001ca96640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f001815dc60 0x7f00180b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45492/0 (socket says 172.21.15.148:45492) 2024-03-04T16:52:33.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.650+0000 7f0017fff640 1 --2- 172.21.15.148:0/2284971140 >> v2:172.21.15.191:3300/0 conn(0x7f00180b5c90 0x7f00181621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.651+0000 7f001d297640 1 -- 172.21.15.148:0/2284971140 >> v2:172.21.15.191:3300/0 conn(0x7f00180b5c90 msgr2=0x7f00181621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:33.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.651+0000 7f001d297640 1 --2- 172.21.15.148:0/2284971140 >> v2:172.21.15.191:3300/0 conn(0x7f00180b5c90 0x7f00181621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.651+0000 7f001d297640 1 -- 172.21.15.148:0/2284971140 >> v1:172.21.15.148:6789/0 conn(0x7f001815dc60 legacy=0x7f00180b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.651+0000 7f001d297640 1 -- 172.21.15.148:0/2284971140 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180b9920 con 0x7f00180b9420 2024-03-04T16:52:33.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.652+0000 7f001d297640 1 --2- 172.21.15.148:0/2284971140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00180b9420 0x7f00180ce150 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f0018067650 tx=0x7f001002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754547c4d9c24e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.652+0000 7f00177fe640 1 -- 172.21.15.148:0/2284971140 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001002cb10 con 0x7f00180b9420 2024-03-04T16:52:33.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.652+0000 7f00177fe640 1 -- 172.21.15.148:0/2284971140 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001002ccb0 con 0x7f00180b9420 2024-03-04T16:52:33.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.652+0000 7f00177fe640 1 -- 172.21.15.148:0/2284971140 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001002ce60 con 0x7f00180b9420 2024-03-04T16:52:33.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.653+0000 7f001ecfa640 1 -- 172.21.15.148:0/2284971140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00180b9420 msgr2=0x7f00180ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.653+0000 7f001ecfa640 1 --2- 172.21.15.148:0/2284971140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00180b9420 0x7f00180ce150 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f0018067650 tx=0x7f001002bf70 comp rx=0 tx=0).stop 2024-03-04T16:52:33.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.653+0000 7f001ecfa640 1 -- 172.21.15.148:0/2284971140 shutdown_connections 2024-03-04T16:52:33.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.653+0000 7f001ecfa640 1 --2- 172.21.15.148:0/2284971140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00180b9420 0x7f00180ce150 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.653+0000 7f001ecfa640 1 --2- 172.21.15.148:0/2284971140 >> v2:172.21.15.191:3300/0 conn(0x7f00180b5c90 0x7f00181621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.654+0000 7f001ecfa640 1 -- 172.21.15.148:0/2284971140 >> 172.21.15.148:0/2284971140 conn(0x7f00180d0580 msgr2=0x7f00180d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:33.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.655+0000 7f001ecfa640 1 -- 172.21.15.148:0/2284971140 shutdown_connections 2024-03-04T16:52:33.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.655+0000 7f001ecfa640 1 -- 172.21.15.148:0/2284971140 wait complete. 2024-03-04T16:52:33.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.656+0000 7f001ecfa640 1 Processor -- start 2024-03-04T16:52:33.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.657+0000 7f001ecfa640 1 -- start start 2024-03-04T16:52:33.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.658+0000 7f001ecfa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.658+0000 7f001ecfa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00180b9420 0x7f00181fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.658+0000 7f001ecfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 0x7f00181fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.148:59858/0 (socket says 172.21.15.148:59858) 2024-03-04T16:52:33.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ecfa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180d1f20 con 0x7f001815dc60 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ecfa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180d2290 con 0x7f00180b9420 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ecfa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001815e040 con 0x7f00180b5c90 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 -- 172.21.15.148:0/1863109279 learned_addr learned my addr 172.21.15.148:0/1863109279 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f0017fff640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00180b9420 0x7f00181fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001d297640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 0x7f00181fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 -- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00180b9420 msgr2=0x7f00181fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00180b9420 0x7f00181fb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 -- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 msgr2=0x7f00181fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 0x7f00181fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 -- 172.21.15.148:0/1863109279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180b2dc0 con 0x7f00180b5c90 2024-03-04T16:52:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001d297640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 0x7f00181fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.659+0000 7f001ca96640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f000c00c3c0 tx=0x7f000c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.660+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0074e0 con 0x7f00180b5c90 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.660+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c007680 con 0x7f00180b5c90 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.660+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00180b3060 con 0x7f00180b5c90 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.660+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00180b3620 con 0x7f00180b5c90 2024-03-04T16:52:33.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.660+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c004a00 con 0x7f00180b5c90 2024-03-04T16:52:33.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.661+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff8005290 con 0x7f00180b5c90 2024-03-04T16:52:33.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.662+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f000c01a020 con 0x7f00180b5c90 2024-03-04T16:52:33.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.662+0000 7f00157fa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:33.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.663+0000 7f0017fff640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:33.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.663+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f000c0899d0 con 0x7f00180b5c90 2024-03-04T16:52:33.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.663+0000 7f0017fff640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f00180c6e60 tx=0x7f0004009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:33.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.670+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f000c058730 con 0x7f00180b5c90 2024-03-04T16:52:33.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.987+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efff8005850 con 0x7f00180b5c90 2024-03-04T16:52:33.967 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-04T16:52:33.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.988+0000 7f00157fa640 1 -- 172.21.15.148:0/1863109279 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f000c058030 con 0x7f00180b5c90 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 msgr2=0x7efffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f00180c6e60 tx=0x7f0004009190 comp rx=0 tx=0).stop 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 msgr2=0x7f00180c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f000c00c3c0 tx=0x7f000c00c880 comp rx=0 tx=0).stop 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 shutdown_connections 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001815dc60 0x7f00181fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00180b9420 0x7f00181fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.992+0000 7f001ecfa640 1 --2- 172.21.15.148:0/1863109279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00180b5c90 0x7f00180c8b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:33.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.993+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 >> 172.21.15.148:0/1863109279 conn(0x7f00180d0580 msgr2=0x7f001806c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:33.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.993+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 shutdown_connections 2024-03-04T16:52:33.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:33.993+0000 7f001ecfa640 1 -- 172.21.15.148:0/1863109279 wait complete. 2024-03-04T16:52:33.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T16:52:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.182+0000 7fa2e3acb640 1 Processor -- start 2024-03-04T16:52:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.182+0000 7fa2e3acb640 1 -- start start 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.183+0000 7fa2e3acb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.183+0000 7fa2e3acb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 0x7fa2dc0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.183+0000 7fa2e3acb640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc0cf740 con 0x7fa2dc0d20c0 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.183+0000 7fa2e3acb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc06c5a0 con 0x7fa2dc161390 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.183+0000 7fa2e3acb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc06d3e0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e1867640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e1867640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38274/0 (socket says 172.21.15.148:38274) 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e1867640 1 -- 172.21.15.148:0/2595923380 learned_addr learned my addr 172.21.15.148:0/2595923380 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e2068640 1 --2- 172.21.15.148:0/2595923380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 0x7fa2dc0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e2068640 1 -- 172.21.15.148:0/2595923380 >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 msgr2=0x7fa2dc06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e2068640 1 --2- 172.21.15.148:0/2595923380 >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e2068640 1 -- 172.21.15.148:0/2595923380 >> v1:172.21.15.191:6789/0 conn(0x7fa2dc0d20c0 legacy=0x7fa2dc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e2068640 1 -- 172.21.15.148:0/2595923380 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc06c7d0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.184+0000 7fa2e1867640 1 --2- 172.21.15.148:0/2595923380 >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:52:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.187+0000 7fa2e2068640 1 --2- 172.21.15.148:0/2595923380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 0x7fa2dc0cc0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0097a0 tx=0x7fa2cc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f357d220498672 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.187+0000 7fa2e0865640 1 -- 172.21.15.148:0/2595923380 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc035020 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.188+0000 7fa2e0865640 1 -- 172.21.15.148:0/2595923380 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc031830 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.188+0000 7fa2e0865640 1 -- 172.21.15.148:0/2595923380 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc031c00 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.190+0000 7fa2e3acb640 1 -- 172.21.15.148:0/2595923380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 msgr2=0x7fa2dc0cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.190+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/2595923380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 0x7fa2dc0cc0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0097a0 tx=0x7fa2cc02b8d0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.191+0000 7fa2e3acb640 1 -- 172.21.15.148:0/2595923380 shutdown_connections 2024-03-04T16:52:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.191+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/2595923380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0cedf0 0x7fa2dc0cc0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.191+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/2595923380 >> v2:172.21.15.148:3300/0 conn(0x7fa2dc161390 0x7fa2dc06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.191+0000 7fa2e3acb640 1 -- 172.21.15.148:0/2595923380 >> 172.21.15.148:0/2595923380 conn(0x7fa2dc066dc0 msgr2=0x7fa2dc0671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.195+0000 7fa2e3acb640 1 -- 172.21.15.148:0/2595923380 shutdown_connections 2024-03-04T16:52:34.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.196+0000 7fa2e3acb640 1 -- 172.21.15.148:0/2595923380 wait complete. 2024-03-04T16:52:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.197+0000 7fa2e3acb640 1 Processor -- start 2024-03-04T16:52:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.197+0000 7fa2e3acb640 1 -- start start 2024-03-04T16:52:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0d20c0 0x7fa2dc1e7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2dc161390 0x7fa2dc0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc06d070 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc06cd90 con 0x7fa2dc161390 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e3acb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc161770 con 0x7fa2dc0d20c0 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.198+0000 7fa2e1867640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38286/0 (socket says 172.21.15.148:38286) 2024-03-04T16:52:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 -- 172.21.15.148:0/3493657689 learned_addr learned my addr 172.21.15.148:0/3493657689 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 -- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0d20c0 msgr2=0x7fa2dc1e7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0d20c0 0x7fa2dc1e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 -- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2dc161390 msgr2=0x7fa2dc0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2dc161390 0x7fa2dc0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 -- 172.21.15.148:0/3493657689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc1e93e0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2e1867640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400efc0 tx=0x7fa2d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.199+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400ce60 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.200+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc1e68c0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.200+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc1e6e80 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.200+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4019070 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.201+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc074f60 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.202+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40146a0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.203+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa2d4005d10 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.203+0000 7fa2d27fc640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 0x7fa2c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.204+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fa2d4056650 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.205+0000 7fa2e1066640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 0x7fa2c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.206+0000 7fa2e1066640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 0x7fa2c006d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8000f60 tx=0x7fa2d80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.211+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2d4010040 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.522+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2dc0c9390 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.522+0000 7fa2d27fc640 1 -- 172.21.15.148:0/3493657689 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2d4058cb0 con 0x7fa2dc0cedf0 2024-03-04T16:52:34.501 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-04T16:52:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 msgr2=0x7fa2c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 0x7fa2c006d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8000f60 tx=0x7fa2d80079b0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 msgr2=0x7fa2dc15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400efc0 tx=0x7fa2d400c490 comp rx=0 tx=0).stop 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 shutdown_connections 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7fa2c006b4d0 0x7fa2c006d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2dc161390 0x7fa2dc0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2dc0d20c0 0x7fa2dc1e7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 --2- 172.21.15.148:0/3493657689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2dc0cedf0 0x7fa2dc15cc90 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 >> 172.21.15.148:0/3493657689 conn(0x7fa2dc066dc0 msgr2=0x7fa2dc0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 shutdown_connections 2024-03-04T16:52:34.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.525+0000 7fa2e3acb640 1 -- 172.21.15.148:0/3493657689 wait complete. 2024-03-04T16:52:34.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T16:52:34.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.699+0000 7f1d482f1640 1 Processor -- start 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.700+0000 7f1d482f1640 1 -- start start 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d482f1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d400cdd30 0x7f1d400ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d482f1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d400ce850 0x7f1d4006e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d482f1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d40162730 con 0x7f1d4015fcb0 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d482f1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d40160090 con 0x7f1d400ce850 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d482f1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d40162a10 con 0x7f1d400cdd30 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d4688e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d400ce850 0x7f1d4006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d4688e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d400ce850 0x7f1d4006e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38288/0 (socket says 172.21.15.148:38288) 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.701+0000 7f1d4688e640 1 -- 172.21.15.148:0/2656497684 learned_addr learned my addr 172.21.15.148:0/2656497684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.702+0000 7f1d4588c640 1 --2- 172.21.15.148:0/2656497684 >> v2:172.21.15.191:3300/0 conn(0x7f1d400cdd30 0x7f1d400ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.702+0000 7f1d4688e640 1 -- 172.21.15.148:0/2656497684 >> v2:172.21.15.191:3300/0 conn(0x7f1d400cdd30 msgr2=0x7f1d400ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T16:52:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.702+0000 7f1d4688e640 1 --2- 172.21.15.148:0/2656497684 >> v2:172.21.15.191:3300/0 conn(0x7f1d400cdd30 0x7f1d400ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.702+0000 7f1d4688e640 1 -- 172.21.15.148:0/2656497684 >> v1:172.21.15.191:6789/0 conn(0x7f1d4015fcb0 legacy=0x7f1d400cd620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T16:52:34.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.702+0000 7f1d4688e640 1 -- 172.21.15.148:0/2656497684 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40162cf0 con 0x7f1d400ce850 2024-03-04T16:52:34.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.703+0000 7f1d4688e640 1 --2- 172.21.15.148:0/2656497684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400ce850 0x7f1d4006e220 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f1d4006d9d0 tx=0x7f1d3802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91cd8d2a54de40 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.703+0000 7f1d4508b640 1 -- 172.21.15.148:0/2656497684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3802c9f0 con 0x7f1d400ce850 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.703+0000 7f1d4508b640 1 -- 172.21.15.148:0/2656497684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3802cb90 con 0x7f1d400ce850 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.704+0000 7f1d4508b640 1 -- 172.21.15.148:0/2656497684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380343f0 con 0x7f1d400ce850 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.704+0000 7f1d482f1640 1 -- 172.21.15.148:0/2656497684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400ce850 msgr2=0x7f1d4006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.704+0000 7f1d482f1640 1 --2- 172.21.15.148:0/2656497684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400ce850 0x7f1d4006e220 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f1d4006d9d0 tx=0x7f1d3802be50 comp rx=0 tx=0).stop 2024-03-04T16:52:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 -- 172.21.15.148:0/2656497684 shutdown_connections 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 --2- 172.21.15.148:0/2656497684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400ce850 0x7f1d4006e220 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 --2- 172.21.15.148:0/2656497684 >> v2:172.21.15.191:3300/0 conn(0x7f1d400cdd30 0x7f1d400ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 -- 172.21.15.148:0/2656497684 >> 172.21.15.148:0/2656497684 conn(0x7f1d400cc4d0 msgr2=0x7f1d400cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 -- 172.21.15.148:0/2656497684 shutdown_connections 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.705+0000 7f1d482f1640 1 -- 172.21.15.148:0/2656497684 wait complete. 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.706+0000 7f1d482f1640 1 Processor -- start 2024-03-04T16:52:34.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.706+0000 7f1d482f1640 1 -- start start 2024-03-04T16:52:34.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.707+0000 7f1d482f1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.707+0000 7f1d482f1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d400ce850 0x7f1d400b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.707+0000 7f1d482f1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015fcb0 0x7f1d4014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d482f1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cee00 con 0x7f1d400cdd30 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d482f1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cec80 con 0x7f1d4015fcb0 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d482f1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4006e760 con 0x7f1d400ce850 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38298/0 (socket says 172.21.15.148:38298) 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 -- 172.21.15.148:0/4022508197 learned_addr learned my addr 172.21.15.148:0/4022508197 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4688e640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015fcb0 0x7f1d4014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4588c640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d400ce850 0x7f1d400b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 -- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d400ce850 msgr2=0x7f1d400b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d400ce850 0x7f1d400b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 -- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015fcb0 msgr2=0x7f1d4014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015fcb0 0x7f1d4014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:34.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 -- 172.21.15.148:0/4022508197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4014d550 con 0x7f1d400cdd30 2024-03-04T16:52:34.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.708+0000 7f1d4608d640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b410 tx=0x7f1d3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.709+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30015020 con 0x7f1d400cdd30 2024-03-04T16:52:34.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.709+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30011830 con 0x7f1d400cdd30 2024-03-04T16:52:34.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.710+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30011c20 con 0x7f1d400cdd30 2024-03-04T16:52:34.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.710+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4014d7f0 con 0x7f1d400cdd30 2024-03-04T16:52:34.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.710+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4014ddb0 con 0x7f1d400cdd30 2024-03-04T16:52:34.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.712+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1d30018020 con 0x7f1d400cdd30 2024-03-04T16:52:34.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.712+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40074f60 con 0x7f1d400cdd30 2024-03-04T16:52:34.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.712+0000 7f1d36ffd640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 0x7f1d2406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:52:34.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.713+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f1d30089c80 con 0x7f1d400cdd30 2024-03-04T16:52:34.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.713+0000 7f1d4588c640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 0x7f1d2406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:52:34.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.713+0000 7f1d4588c640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 0x7f1d2406da90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d40071f00 tx=0x7f1d3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:52:34.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:34.722+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d3001f070 con 0x7f1d400cdd30 2024-03-04T16:52:35.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.032+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d40165000 con 0x7f1d400cdd30 2024-03-04T16:52:35.011 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-04T16:52:35.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.032+0000 7f1d36ffd640 1 -- 172.21.15.148:0/4022508197 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d300582e0 con 0x7f1d400cdd30 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 msgr2=0x7f1d2406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 0x7f1d2406da90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d40071f00 tx=0x7f1d3c008040 comp rx=0 tx=0).stop 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 msgr2=0x7f1d400b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b410 tx=0x7f1d3000b8d0 comp rx=0 tx=0).stop 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 shutdown_connections 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:6832/2037319536,v1:172.21.15.148:6833/2037319536] conn(0x7f1d2406b5d0 0x7f1d2406da90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015fcb0 0x7f1d4014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d400ce850 0x7f1d400b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 --2- 172.21.15.148:0/4022508197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d400cdd30 0x7f1d400b3af0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:52:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 >> 172.21.15.148:0/4022508197 conn(0x7f1d400cc4d0 msgr2=0x7f1d401637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:52:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.035+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 shutdown_connections 2024-03-04T16:52:35.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-04T16:52:35.039+0000 7f1d482f1640 1 -- 172.21.15.148:0/4022508197 wait complete. 2024-03-04T16:52:35.041 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T16:52:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T16:52:35.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-04T16:52:35.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T16:52:35.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T16:52:35.099 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-04T16:52:35.141 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-04T16:52:35.141 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T16:52:35.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056373,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3630 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 172.21.15.148:6789:/test /tmp/tmp.1jHsm6tBGxtest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=d4b3ed31-a94c-42d0-968e-1d6a4e199a28,acl 0 0 2024-03-04T16:52:35.205 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-04T16:52:35.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T16:52:35.205 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-04T16:52:35.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T16:52:35.267 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-04T16:52:35.268 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T16:52:35.268 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T16:52:35.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056376,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2586 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-04T16:52:35.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T16:52:35.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T16:52:35.372 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-04T16:52:35.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T16:52:35.452 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-04T16:52:35.452 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T16:52:35.452 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T16:52:35.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056376,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2586 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-04T16:52:35.513 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T16:52:35.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T16:52:35.513 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-04T16:52:35.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T16:52:35.570 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-04T16:52:37.180 INFO:tasks.ceph.osd.0:Sent signal 15 2024-03-04T16:52:37.181 INFO:tasks.ceph.osd.1:Sent signal 15 2024-03-04T16:52:37.181 INFO:tasks.ceph.osd.2:Sent signal 15 2024-03-04T16:52:37.182 INFO:tasks.ceph.osd.3:Sent signal 15 2024-03-04T16:52:37.182 INFO:tasks.ceph.osd.4:Sent signal 15 2024-03-04T16:52:37.182 INFO:tasks.ceph.osd.6:Sent signal 15 2024-03-04T16:52:37.182 INFO:tasks.ceph.osd.7:Sent signal 15 2024-03-04T16:52:37.183 INFO:tasks.ceph.mds.a:Sent signal 15 2024-03-04T16:52:37.183 INFO:tasks.ceph.mds.c:Sent signal 15 2024-03-04T16:52:37.183 INFO:tasks.ceph.mds.b:Sent signal 15 2024-03-04T16:52:37.183 INFO:tasks.ceph.mds.d:Sent signal 15 2024-03-04T16:52:37.183 INFO:tasks.ceph.mon.a:Sent signal 15 2024-03-04T16:52:37.184 INFO:tasks.ceph.mon.b:Sent signal 15 2024-03-04T16:52:37.184 INFO:tasks.ceph.mon.c:Sent signal 15 2024-03-04T16:52:37.184 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-03-04T16:52:37.184 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-03-04T16:52:37.185 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f382c788640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 38100) UID: 0 2024-03-04T16:52:37.185 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f382c788640 -1 osd.4 42 *** Got signal Terminated *** 2024-03-04T16:52:37.185 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f382c788640 -1 osd.4 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.186 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f6881ba1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 38104) UID: 0 2024-03-04T16:52:37.186 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f6881ba1640 -1 osd.6 42 *** Got signal Terminated *** 2024-03-04T16:52:37.186 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f6881ba1640 -1 osd.6 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.186 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f1dfcccf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 38106) UID: 0 2024-03-04T16:52:37.187 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f1dfcccf640 -1 osd.7 42 *** Got signal Terminated *** 2024-03-04T16:52:37.187 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-04T16:52:37.204+0000 7f1dfcccf640 -1 osd.7 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.187 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-04T16:52:37.205+0000 7fea1d89f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 44932) UID: 0 2024-03-04T16:52:37.187 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-04T16:52:37.205+0000 7fea1d89f640 -1 mds.b *** got signal Terminated *** 2024-03-04T16:52:37.187 INFO:tasks.ceph.mds.d.smithi191.stderr:2024-03-04T16:52:37.205+0000 7f20e03ed640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 44933) UID: 0 2024-03-04T16:52:37.188 INFO:tasks.ceph.mds.d.smithi191.stderr:2024-03-04T16:52:37.205+0000 7f20e03ed640 -1 mds.d *** got signal Terminated *** 2024-03-04T16:52:37.188 INFO:tasks.ceph.mon.b.smithi191.stderr:2024-03-04T16:52:37.206+0000 7f943dc73640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 37531) UID: 0 2024-03-04T16:52:37.188 INFO:tasks.ceph.mon.b.smithi191.stderr:2024-03-04T16:52:37.206+0000 7f943dc73640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-03-04T16:52:37.188 INFO:tasks.ceph.mon.c.smithi191.stderr:2024-03-04T16:52:37.206+0000 7f67814bc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 37533) UID: 0 2024-03-04T16:52:37.188 INFO:tasks.ceph.mon.c.smithi191.stderr:2024-03-04T16:52:37.206+0000 7f67814bc640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-03-04T16:52:37.189 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:52:37.202+0000 7fe45f924640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 38098) UID: 0 2024-03-04T16:52:37.189 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:52:37.202+0000 7fe45f924640 -1 osd.0 42 *** Got signal Terminated *** 2024-03-04T16:52:37.189 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-04T16:52:37.202+0000 7fe45f924640 -1 osd.0 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fb0f0776640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 38106) UID: 0 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fb0f0776640 -1 osd.1 42 *** Got signal Terminated *** 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fb0f0776640 -1 osd.1 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fea35801640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 38104) UID: 0 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fea35801640 -1 osd.2 42 *** Got signal Terminated *** 2024-03-04T16:52:37.190 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-04T16:52:37.203+0000 7fea35801640 -1 osd.2 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.191 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:52:37.204+0000 7f19b5ca4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 38108) UID: 0 2024-03-04T16:52:37.191 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:52:37.204+0000 7f19b5ca4640 -1 osd.3 42 *** Got signal Terminated *** 2024-03-04T16:52:37.191 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-04T16:52:37.204+0000 7f19b5ca4640 -1 osd.3 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:52:37.191 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T16:52:37.205+0000 7f9bc7e2b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 52790) UID: 0 2024-03-04T16:52:37.191 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-04T16:52:37.205+0000 7f9bc7e2b640 -1 mds.a *** got signal Terminated *** 2024-03-04T16:52:37.192 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T16:52:37.205+0000 7fa25d993640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 52791) UID: 0 2024-03-04T16:52:37.192 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-03-04T16:52:37.205+0000 7fa25d993640 -1 mds.c *** got signal Terminated *** 2024-03-04T16:52:37.192 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-03-04T16:52:37.205+0000 7fd7dfc5b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 37826) UID: 0 2024-03-04T16:52:37.192 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-03-04T16:52:37.205+0000 7fd7dfc5b640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T16:52:37.192 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-04T16:52:37.206+0000 7fa5e76e1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i y (PID: 37863) UID: 0 2024-03-04T16:52:37.387 INFO:tasks.ceph.mgr.x.smithi191.stderr:daemon-helper: command crashed with signal 15 2024-03-04T16:52:38.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:52:38.640 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:54:44.383 INFO:teuthology.orchestra.run.smithi148.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.2.log.1: 2024-03-04T16:54:44.383 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: stdin: file size changed while zipping 2024-03-04T16:55:16.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:55:16.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:55:46.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:55:46.755 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:56:16.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:56:16.825 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:56:46.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:56:46.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:57:17.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:57:17.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:57:47.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:57:47.195 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:58:17.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:58:17.348 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:58:47.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:58:47.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:59:17.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:59:17.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:59:48.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T16:59:48.035 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:00:18.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:00:18.215 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:00:48.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:00:48.350 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:01:18.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:01:18.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:01:48.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:01:48.626 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:02:18.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:02:18.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:02:48.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:02:48.985 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:03:19.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:03:19.124 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:03:49.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:03:49.254 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:04:19.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:04:19.386 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:04:49.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:04:49.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:05:19.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:05:19.882 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:05:50.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:05:50.042 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:06:20.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:06:20.171 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:06:50.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:06:50.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:07:20.602 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:07:20.606 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:07:50.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:07:50.743 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:08:20.889 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:08:20.893 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:08:50.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:08:50.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:09:21.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:09:21.108 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:09:51.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:09:51.236 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:10:21.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:10:21.417 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:10:51.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:10:51.562 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:11:21.712 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:11:21.716 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:11:51.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:11:51.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:12:21.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:12:21.984 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:12:52.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:12:52.121 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:13:22.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:13:22.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:13:52.492 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:13:52.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:14:22.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:14:22.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:14:52.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:14:52.712 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:15:23.038 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:15:23.042 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:15:53.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:15:53.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:16:23.324 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:16:23.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:16:53.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:16:53.662 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:17:23.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:17:23.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:17:53.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:17:53.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:18:24.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:18:24.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:18:54.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:18:54.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:19:24.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:19:24.209 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:19:54.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:19:54.364 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:20:24.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:20:24.522 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:20:54.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:20:54.660 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:21:24.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:21:24.806 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:21:54.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:21:54.951 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:22:25.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:22:25.089 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:22:55.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:22:55.228 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:23:25.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:23:25.364 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:23:55.594 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:23:55.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:24:25.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:24:25.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:24:55.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:24:55.834 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:25:26.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:25:26.120 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:25:56.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:25:56.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:26:26.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:26:26.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:26:57.100 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:26:57.103 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:27:27.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:27:27.189 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:27:57.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:27:57.349 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:28:27.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:28:27.494 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:28:57.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:28:57.634 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:29:27.704 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:29:27.714 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:29:57.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:29:57.901 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:30:28.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:30:28.046 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:30:58.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:30:58.409 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:31:28.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:31:28.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:31:58.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:31:58.663 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:32:29.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:32:29.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:32:59.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:32:59.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:33:29.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:33:29.206 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:33:59.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:33:59.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:34:29.507 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:34:29.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:34:59.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:34:59.640 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:35:29.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:35:29.702 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:35:59.787 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:35:59.791 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:36:29.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:36:29.929 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:37:00.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:37:00.076 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:37:30.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:37:30.205 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:38:00.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:38:00.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:38:30.564 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:38:30.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:39:00.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:39:00.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:39:30.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:39:30.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:40:00.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:40:01.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:40:31.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:40:31.141 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:41:01.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:41:01.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:41:31.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:41:31.350 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:42:01.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:42:01.424 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:42:31.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:42:31.501 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:43:01.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:43:01.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:43:31.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:43:31.943 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:44:02.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:44:02.023 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:44:32.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:44:32.096 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:45:02.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:45:02.166 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:45:32.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:45:32.324 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:46:02.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:46:02.478 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:46:32.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:46:32.607 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:47:02.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:47:02.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:47:33.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:47:33.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:48:03.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:48:03.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:48:33.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:48:33.430 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:49:03.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:49:03.639 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:49:33.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:49:33.769 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:50:03.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:50:03.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:50:33.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:50:33.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:51:04.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:51:04.090 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:51:34.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:51:34.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:52:04.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:52:04.361 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:52:34.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:52:34.550 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:53:04.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:53:04.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:53:34.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:53:34.866 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:54:05.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:54:05.012 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:54:35.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:54:35.134 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:55:05.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:55:05.205 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:55:35.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:55:35.385 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:56:05.512 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:56:05.516 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:56:35.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:56:35.628 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:57:05.738 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:57:05.741 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:57:35.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:57:35.886 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:58:06.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:58:06.052 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:58:36.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:58:36.197 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:59:06.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:59:06.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:59:36.496 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T17:59:36.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:00:06.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:00:06.648 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:00:36.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:00:36.795 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:01:06.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:01:06.952 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:01:37.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:01:37.015 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:02:07.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:02:07.076 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:02:37.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:02:37.216 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:03:07.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:03:07.358 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:03:37.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:03:37.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:04:07.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:04:07.625 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:04:37.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:04:37.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:05:07.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:05:07.831 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:05:37.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:05:37.986 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:06:08.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:06:08.066 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:06:38.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:06:38.305 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:07:08.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:07:08.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:07:38.594 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:07:38.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:08:08.734 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:08:08.738 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:08:38.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:08:38.866 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:09:09.016 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:09:09.020 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:09:39.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:09:39.175 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:10:09.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:10:09.318 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:10:39.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:10:39.569 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:11:09.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:11:09.901 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:11:40.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:11:40.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:12:10.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:12:10.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:12:40.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:12:40.347 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:13:10.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:13:10.477 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:13:40.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:13:40.640 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:14:10.777 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:14:10.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:14:41.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:14:41.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:15:11.252 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:15:11.255 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:15:41.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:15:41.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:16:11.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:16:11.456 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:16:41.554 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:16:41.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:17:11.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:17:11.706 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:17:42.025 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:17:42.028 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:18:12.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:18:12.176 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:18:42.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:18:42.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:19:12.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:19:12.351 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:19:42.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:19:42.478 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:20:12.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:20:12.623 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:20:42.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:20:42.744 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:21:12.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:21:12.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:21:42.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:21:42.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:22:13.084 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:22:13.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:22:43.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:22:43.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:23:13.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:23:13.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:23:43.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:23:43.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:24:13.782 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:24:13.785 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:24:43.852 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:24:43.855 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:25:13.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:25:13.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:25:44.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:25:44.138 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:26:14.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:26:14.276 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:26:44.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:26:44.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:27:14.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:27:14.523 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:27:44.687 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:27:44.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:28:14.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:28:14.770 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:28:44.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:28:44.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:29:14.942 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:29:14.946 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:29:45.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:29:45.075 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:30:15.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:30:15.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:30:45.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:30:45.368 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:31:15.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:31:15.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:31:45.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:31:45.673 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:32:15.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:32:15.793 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:32:45.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:32:45.930 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:33:16.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:33:16.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:33:46.185 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:33:46.188 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:34:16.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:34:16.258 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:34:46.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:34:46.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:35:16.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:35:16.457 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:35:46.532 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:35:46.535 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:36:16.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:36:16.607 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:36:46.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:36:46.761 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:37:16.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:37:16.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:37:47.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:37:47.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:38:17.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:38:17.295 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:38:47.445 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:38:47.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:39:17.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:39:17.594 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:39:47.753 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:39:47.757 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:40:17.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:40:17.892 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:40:48.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:40:48.066 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:41:18.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:41:18.212 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:41:48.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:41:48.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:42:18.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:42:18.446 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:42:48.724 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:42:48.728 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:43:18.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:43:18.875 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:43:49.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:43:49.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:44:19.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:44:19.099 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:44:49.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:44:49.309 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:45:19.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:45:19.435 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:45:49.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:45:49.564 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:46:19.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:46:19.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:46:50.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:46:50.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:47:20.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:47:20.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:47:50.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:47:50.258 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:48:20.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:48:20.435 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:48:50.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:48:50.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:49:20.797 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:49:20.800 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:49:50.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:49:50.928 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:50:21.054 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:50:21.057 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:50:51.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:50:51.202 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:51:21.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:51:21.349 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:51:51.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:51:51.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:52:21.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:52:21.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:52:51.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:52:51.767 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:53:21.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:53:21.971 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:53:52.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:53:52.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:54:22.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:54:22.268 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:54:52.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:54:52.407 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:55:22.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:55:22.537 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:55:52.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:55:52.642 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:56:22.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:56:22.794 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:56:53.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:56:53.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:57:23.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:57:23.175 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:57:53.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:57:53.246 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:58:23.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:58:23.391 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:58:53.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:58:53.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:59:23.688 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:59:23.692 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:59:53.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T18:59:53.837 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:00:23.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:00:23.984 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:00:54.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:00:54.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:01:24.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:01:24.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:01:54.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:01:54.390 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:02:24.541 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:02:24.544 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:02:54.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:02:54.698 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:03:24.833 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:03:24.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:03:55.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:03:55.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:04:25.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:04:25.175 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:04:55.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:04:55.329 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:05:25.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:05:25.465 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:05:55.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:05:55.584 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:06:25.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:06:25.731 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:06:55.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:06:55.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:07:26.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:07:26.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:07:56.132 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:07:56.135 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:08:26.262 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:08:26.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:08:56.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:08:56.434 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:09:26.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:09:26.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:09:56.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:09:56.832 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:10:26.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:10:26.966 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:10:57.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:10:57.122 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:11:27.247 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:11:27.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:11:57.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:11:57.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:12:27.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:12:27.560 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:12:57.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:12:57.706 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:13:27.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:13:27.851 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:13:57.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:13:57.915 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:14:28.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:14:28.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:14:58.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:14:58.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:15:28.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:15:28.413 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:15:58.539 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:15:58.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:16:28.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:16:28.672 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:16:58.799 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:16:58.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:17:28.959 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:17:28.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:17:59.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:17:59.262 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:18:29.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:18:29.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:18:59.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:18:59.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:19:29.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:19:29.559 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:19:59.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:19:59.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:20:29.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:20:29.862 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:21:00.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:21:00.040 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:21:30.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:21:30.203 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:22:00.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:22:00.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:22:30.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:22:30.518 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:23:00.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:23:00.679 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:23:30.837 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:23:30.841 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:24:00.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:24:00.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:24:31.145 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:24:31.149 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:25:01.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:25:01.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:25:31.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:25:31.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:26:01.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:26:01.586 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:26:31.705 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:26:31.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:27:01.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:27:01.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:27:31.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:27:31.979 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:28:02.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:28:02.139 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:28:32.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:28:32.288 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:29:02.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:29:02.432 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:29:32.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:29:32.580 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:30:02.697 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:30:02.700 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:30:32.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:30:32.847 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:31:03.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:31:03.069 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:31:33.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:31:33.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:32:03.349 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:32:03.352 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:32:33.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:32:33.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:33:03.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:33:03.646 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:33:33.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:33:33.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:34:03.930 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:34:03.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:34:34.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:34:34.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:35:04.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:35:04.190 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:35:34.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:35:34.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:36:04.480 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:36:04.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:36:34.667 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:36:34.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:37:04.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:37:04.814 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:37:34.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:37:34.954 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:38:05.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:38:05.096 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:38:35.246 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:38:35.249 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:39:05.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:39:05.387 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:39:35.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:39:35.548 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:40:05.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:40:05.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:40:35.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:40:35.833 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:41:05.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:41:05.966 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:41:36.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:41:36.120 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:42:06.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:42:06.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:42:36.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:42:36.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:43:06.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:43:06.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:43:36.732 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:43:36.735 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:44:06.886 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:44:06.889 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:44:37.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:44:37.007 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:45:07.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:45:07.142 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:45:37.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:45:37.274 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:46:07.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:46:07.461 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:46:37.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:46:37.606 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:47:07.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:47:07.743 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:47:37.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:47:37.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:48:07.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:48:07.983 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:48:38.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:48:38.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:49:08.235 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:49:08.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:49:38.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:49:38.361 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:50:08.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:50:08.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:50:38.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:50:38.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:51:08.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:51:08.832 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:51:38.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:51:38.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:52:09.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:52:09.071 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:52:39.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:52:39.209 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:53:09.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:53:09.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:53:39.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:53:39.497 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:54:09.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:54:09.627 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:54:39.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:54:39.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:55:09.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:55:09.935 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:55:40.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:55:40.196 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:56:10.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:56:10.337 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:56:40.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:56:40.482 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:57:10.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:57:10.660 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:57:40.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:57:40.789 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:58:10.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:58:10.943 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:58:41.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:58:41.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:59:11.213 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:59:11.217 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:59:41.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T19:59:41.393 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:00:11.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:00:11.523 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:00:41.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:00:41.662 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:01:11.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:01:11.809 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:01:41.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:01:41.956 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:02:12.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:02:12.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:02:42.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:02:42.261 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:03:12.403 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:03:12.409 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:03:42.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:03:42.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:04:12.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:04:12.724 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:04:42.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:04:42.853 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:05:13.069 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:05:13.072 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:05:43.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:05:43.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:06:13.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:06:13.462 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:06:43.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:06:43.608 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:07:13.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:07:13.752 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:07:43.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:07:43.895 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:08:14.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:08:14.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:08:44.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:08:44.171 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:09:14.319 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:09:14.323 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:09:44.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:09:44.480 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:10:14.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:10:14.609 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:10:44.767 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:10:44.771 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:11:14.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:11:14.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:11:45.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:11:45.035 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:12:15.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:12:15.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:12:45.293 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:12:45.298 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:13:15.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:13:15.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:13:45.586 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:13:45.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:14:15.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:14:15.715 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:14:46.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:14:46.049 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:15:16.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:15:16.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:15:46.354 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:15:46.358 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:16:16.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:16:16.506 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:16:46.672 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:16:46.676 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:17:16.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:17:16.838 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:17:46.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:17:46.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:18:17.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:18:17.091 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:18:47.207 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:18:47.211 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:19:17.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:19:17.348 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:19:47.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:19:47.541 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:20:17.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:20:17.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:20:47.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:20:47.819 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:21:17.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:21:17.957 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:21:48.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:21:48.099 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:22:18.238 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:22:18.242 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:22:48.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:22:48.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:23:18.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:23:18.565 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:23:48.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:23:48.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:24:19.069 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:24:19.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:24:49.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:24:49.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:25:19.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:25:19.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:25:49.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:25:49.732 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:26:19.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:26:19.962 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:26:50.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:26:50.093 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:27:20.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:27:20.228 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:27:50.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:27:50.364 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:28:20.501 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:28:20.506 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:28:50.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:28:50.652 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:29:20.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:29:20.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:29:50.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:29:50.946 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:30:21.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:30:21.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:30:51.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:30:51.218 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:31:21.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:31:21.365 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:31:51.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:31:51.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:32:21.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:32:21.565 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:32:51.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:32:51.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:33:21.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:33:21.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:33:52.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:33:52.060 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:34:22.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:34:22.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:34:52.336 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:34:52.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:35:22.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:35:22.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:35:52.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:35:52.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:36:22.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:36:22.754 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:36:52.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:36:52.893 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:37:23.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:37:23.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:37:53.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:37:53.212 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:38:23.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:38:23.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:38:53.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:38:53.430 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:39:23.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:39:23.745 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:39:53.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:39:53.881 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:40:23.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:40:23.979 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:40:54.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:40:54.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:41:24.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:41:24.294 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:41:54.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:41:54.448 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:42:24.590 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:42:24.593 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:42:54.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:42:54.747 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:43:24.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:43:24.893 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:43:55.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:43:55.030 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:44:25.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:44:25.177 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:44:55.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:44:55.332 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:45:25.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:45:25.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:45:55.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:45:55.609 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:46:25.730 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:46:25.733 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:46:55.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:46:55.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:47:26.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:47:26.050 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:47:56.208 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:47:56.211 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:48:26.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:48:26.359 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:48:56.501 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:48:56.505 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:49:26.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:49:26.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:49:56.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:49:56.795 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:50:26.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:50:26.968 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:50:57.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:50:57.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:51:27.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:51:27.269 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:51:57.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:51:57.431 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:52:27.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:52:27.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:52:57.685 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:52:57.688 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:53:27.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:53:27.816 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:53:57.942 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:53:57.946 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:54:28.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:54:28.101 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:54:58.259 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:54:58.263 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:55:28.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:55:28.399 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:55:58.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:55:58.555 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:56:28.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:56:28.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:56:58.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:56:58.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:57:29.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:57:29.012 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:57:59.213 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:57:59.216 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:58:29.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:58:29.361 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:58:59.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:58:59.508 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:59:29.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:59:29.677 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:59:59.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T20:59:59.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:00:29.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:00:29.958 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:01:00.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:01:00.097 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:01:30.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:01:30.225 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:02:00.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:02:00.361 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:02:30.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:02:30.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:03:00.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:03:00.642 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:03:30.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:03:30.768 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:04:00.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:04:00.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:04:31.107 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:04:31.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:05:01.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:05:01.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:05:31.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:05:31.393 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:06:01.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:06:01.539 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:06:31.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:06:31.693 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:07:01.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:07:01.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:07:32.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:07:32.077 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:08:02.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:08:02.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:08:32.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:08:32.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:09:02.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:09:02.455 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:09:32.602 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:09:32.609 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:10:02.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:10:02.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:10:32.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:10:32.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:11:03.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:11:03.048 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:11:33.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:11:33.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:12:03.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:12:03.565 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:12:33.704 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:12:33.707 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:13:03.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:13:03.853 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:13:33.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:13:33.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:14:04.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:14:04.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:14:34.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:14:34.197 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:15:04.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:15:04.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:15:34.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:15:34.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:16:04.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:16:04.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:16:34.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:16:34.839 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:17:04.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:17:04.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:17:34.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:17:34.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:18:05.030 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:18:05.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:18:35.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:18:35.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:19:05.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:19:05.302 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:19:35.445 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:19:35.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:20:05.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:20:05.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:20:35.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:20:35.649 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:21:05.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:21:05.712 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:21:35.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:21:35.862 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:22:05.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:22:06.003 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:22:36.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:22:36.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:23:06.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:23:06.290 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:23:36.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:23:36.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:24:06.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:24:06.573 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:24:36.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:24:36.803 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:25:07.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:25:07.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:25:37.254 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:25:37.257 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:26:07.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:26:07.378 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:26:37.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:26:37.503 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:27:07.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:27:07.643 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:27:37.810 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:27:37.814 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:28:07.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:28:07.941 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:28:38.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:28:38.089 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:29:08.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:29:08.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:29:38.348 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:29:38.352 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:30:08.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:30:08.505 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:30:38.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:30:38.659 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:31:08.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:31:08.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:31:38.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:31:38.974 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:32:09.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:32:09.101 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:32:39.235 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:32:39.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:33:09.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:33:09.405 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:33:39.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:33:39.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:34:09.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:34:09.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:34:39.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:34:39.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:35:10.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:35:10.038 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:35:40.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:35:40.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:36:10.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:36:10.320 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:36:40.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:36:40.482 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:37:10.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:37:10.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:37:40.768 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:37:40.771 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:38:10.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:38:10.917 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:38:41.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:38:41.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:39:11.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:39:11.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:39:41.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:39:41.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:40:11.539 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:40:11.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:40:41.667 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:40:41.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:41:11.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:41:11.799 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:41:41.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:41:41.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:42:12.090 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:42:12.094 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:42:42.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:42:42.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:43:12.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:43:12.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:43:42.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:43:42.516 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:44:12.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:44:12.675 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:44:42.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:44:42.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:45:12.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:45:12.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:45:43.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:45:43.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:46:13.230 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:46:13.234 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:46:43.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:46:43.402 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:47:13.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:47:13.549 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:47:43.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:47:43.702 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:48:13.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:48:13.847 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:48:43.995 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:48:43.999 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:49:14.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:49:14.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:49:44.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:49:44.285 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:50:14.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:50:14.413 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:50:44.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:50:44.550 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:51:14.684 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:51:14.688 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:51:44.916 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:51:44.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:52:15.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:52:15.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:52:45.305 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:52:45.309 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:53:15.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:53:15.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:53:45.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:53:45.597 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:54:15.732 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:54:15.736 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:54:45.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:54:45.881 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:55:16.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:55:16.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:55:46.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:55:46.171 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:56:16.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:56:16.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:56:46.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:56:46.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:57:16.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:57:16.633 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:57:46.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:57:46.769 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:58:16.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:58:16.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:58:47.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:58:47.036 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:59:17.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:59:17.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:59:47.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T21:59:47.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:00:17.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:00:17.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:00:47.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:00:47.614 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:01:17.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:01:17.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:01:47.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:01:47.918 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:02:18.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:02:18.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:02:48.213 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:02:48.216 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:03:18.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:03:18.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:03:48.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:03:48.523 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:04:18.666 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:04:18.670 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:04:48.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:04:48.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:05:18.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:05:18.938 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:05:49.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:05:49.072 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:06:19.196 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:06:19.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:06:49.325 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:06:49.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:07:19.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:07:19.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:07:49.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:07:49.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:08:19.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:08:19.735 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:08:49.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:08:49.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:09:19.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:09:19.943 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:09:50.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:09:50.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:10:20.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:10:20.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:10:50.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:10:50.369 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:11:20.551 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:11:20.555 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:11:50.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:11:50.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:12:20.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:12:20.789 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:12:50.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:12:50.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:13:21.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:13:21.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:13:51.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:13:51.095 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:14:21.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:14:21.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:14:51.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:14:51.372 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:15:21.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:15:21.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:15:51.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:15:51.681 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:16:21.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:16:21.801 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:16:51.945 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:16:51.948 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:17:22.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:17:22.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:17:52.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:17:52.236 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:18:22.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:18:22.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:18:52.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:18:52.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:19:22.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:19:22.685 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:19:52.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:19:52.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:20:22.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:20:22.973 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:20:53.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:20:53.114 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:21:23.265 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:21:23.269 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:21:53.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:21:53.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:22:23.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:22:23.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:22:53.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:22:53.722 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:23:23.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:23:23.902 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:23:54.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:23:54.069 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:24:24.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:24:24.223 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:24:54.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:24:54.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:25:24.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:25:24.596 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:25:54.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:25:54.736 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:26:24.885 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:26:24.888 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:26:55.030 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:26:55.034 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:27:25.167 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:27:25.170 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:27:55.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:27:55.311 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:28:25.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:28:25.462 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:28:55.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:28:55.630 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:29:25.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:29:25.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:29:55.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:29:55.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:30:26.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:30:26.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:30:56.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:30:56.196 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:31:26.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:31:26.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:31:56.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:31:56.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:32:26.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:32:26.658 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:32:56.799 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:32:56.802 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:33:26.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:33:26.939 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:33:57.107 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:33:57.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:34:27.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:34:27.240 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:34:57.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:34:57.385 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:35:27.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:35:27.546 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:35:57.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:35:57.686 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:36:27.842 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:36:27.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:36:57.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:36:57.984 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:37:28.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:37:28.128 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:37:58.262 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:37:58.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:38:28.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:38:28.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:38:58.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:38:58.555 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:39:28.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:39:28.683 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:39:58.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:39:58.822 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:40:28.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:40:28.967 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:40:59.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:40:59.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:41:29.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:41:29.223 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:41:59.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:41:59.370 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:42:29.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:42:29.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:42:59.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:42:59.659 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:43:29.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:43:29.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:43:59.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:43:59.971 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:44:30.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:44:30.121 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:45:00.278 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:45:00.281 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:45:30.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:45:30.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:46:00.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:46:00.574 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:46:30.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:46:30.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:47:00.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:47:00.838 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:47:30.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:47:30.996 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:48:01.130 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:48:01.134 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:48:31.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:48:31.274 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:49:01.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:49:01.411 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:49:31.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:49:31.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:50:01.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:50:01.668 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:50:31.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:50:31.822 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:51:01.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:51:01.958 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:51:32.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:51:32.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:52:02.259 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:52:02.262 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:52:32.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:52:32.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:53:02.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:53:02.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:53:32.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:53:32.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:54:02.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:54:02.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:54:32.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:54:32.961 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:55:03.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:55:03.086 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:55:33.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:55:33.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:56:03.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:56:03.363 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:56:33.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:56:33.522 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:57:03.665 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:57:03.669 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:57:33.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:57:33.831 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:58:03.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:58:03.976 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:58:34.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:58:34.137 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:59:04.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:59:04.287 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:59:34.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T22:59:34.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:00:04.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:00:04.606 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:00:34.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:00:34.768 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:01:04.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:01:04.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:01:35.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:01:35.109 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:02:05.269 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:02:05.272 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:02:35.457 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:02:35.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:03:05.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:03:05.612 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:03:35.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:03:35.769 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:04:05.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:04:05.916 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:04:36.053 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:04:36.056 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:05:06.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:05:06.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:05:36.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:05:36.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:06:06.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:06:06.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:06:36.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:06:36.607 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:07:06.732 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:07:06.736 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:07:36.870 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:07:36.873 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:08:07.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:08:07.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:08:37.130 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:08:37.133 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:09:07.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:09:07.255 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:09:37.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:09:37.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:10:07.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:10:07.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:10:37.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:10:37.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:11:07.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:11:07.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:11:37.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:11:37.895 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:12:07.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:12:07.961 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:12:38.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:12:38.030 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:13:08.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:13:08.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:13:38.278 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:13:38.281 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:14:08.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:14:08.512 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:14:38.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:14:38.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:15:08.779 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:15:08.783 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:15:38.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:15:38.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:16:09.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:16:09.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:16:39.190 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:16:39.194 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:17:09.352 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:17:09.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:17:39.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:17:39.486 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:18:09.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:18:09.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:18:39.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:18:39.762 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:19:09.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:19:09.919 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:19:39.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:19:39.990 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:20:10.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:20:10.124 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:20:40.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:20:40.262 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:21:10.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:21:10.431 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:21:40.578 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:21:40.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:22:10.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:22:10.741 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:22:40.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:22:40.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:23:11.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:23:11.049 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:23:41.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:23:41.203 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:24:11.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:24:11.332 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:24:41.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:24:41.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:25:11.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:25:11.607 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:25:41.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:25:41.749 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:26:11.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:26:11.910 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:26:42.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:26:42.046 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:27:12.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:27:12.190 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:27:42.348 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:27:42.352 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:28:12.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:28:12.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:28:42.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:28:42.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:29:12.787 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:29:12.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:29:42.916 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:29:42.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:30:13.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:30:13.066 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:30:43.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:30:43.203 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:31:13.337 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:31:13.341 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:31:43.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:31:43.462 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:32:13.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:32:13.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:32:43.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:32:43.754 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:33:13.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:33:13.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:33:44.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:33:44.043 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:34:14.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:34:14.195 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:34:44.352 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:34:44.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:35:14.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:35:14.494 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:35:44.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:35:44.642 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:36:14.782 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:36:14.785 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:36:45.061 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:36:45.065 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:37:15.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:37:15.208 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:37:45.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:37:45.308 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:38:15.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:38:15.437 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:38:45.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:38:45.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:39:15.726 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:39:15.729 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:39:46.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:39:46.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:40:16.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:40:16.168 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:40:46.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:40:46.306 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:41:16.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:41:16.430 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:41:46.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:41:46.575 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:42:16.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:42:16.726 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:42:46.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:42:46.870 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:43:17.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:43:17.015 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:43:47.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:43:47.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:44:17.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:44:17.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:44:47.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:44:47.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:45:17.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:45:17.564 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:45:47.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:45:47.684 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:46:17.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:46:17.834 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:46:47.978 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:46:47.982 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:47:18.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:47:18.119 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:47:48.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:47:48.247 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:48:18.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:48:18.393 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:48:48.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:48:48.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:49:18.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:49:18.668 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:49:48.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:49:48.796 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:50:18.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:50:18.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:50:49.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:50:49.077 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:51:19.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:51:19.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:51:49.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:51:49.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:52:19.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:52:19.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:52:49.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:52:49.666 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:53:19.833 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:53:19.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:53:49.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:53:49.976 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:54:20.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:54:20.116 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:54:50.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:54:50.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:55:20.402 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:55:20.405 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:55:50.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:55:50.565 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:56:20.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:56:20.747 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:56:50.907 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:56:50.910 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:57:21.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:57:21.050 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:57:51.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:57:51.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:58:21.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:58:21.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:58:51.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:58:51.520 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:59:21.647 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:59:21.650 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:59:51.776 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T23:59:51.779 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-05T00:00:21.889 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-05T00:00:21.892 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-05T00:00:52.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-05T00:00:52.047 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-05T00:01:04.296 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-05T00:01:04.297 DEBUG:teuthology.task.console_log:Killing console logger for smithi148 2024-03-05T00:01:04.298 DEBUG:teuthology.task.console_log:Killing console logger for smithi191 2024-03-05T00:01:04.298 DEBUG:teuthology.exit:Finished running handlers